High throughput communication system

ABSTRACT

A high throughput communication apparatus which provides low frame error rates (FER). Error checking encoder and decoders which each comprise a plurality of short blocklength error checking encoders or decoders, respectively, in parallel, coupled through common incremental redundancy. Short-blocklength codes are utilized to achieve communication capacity with incremental redundancy. The system can transmit and decode a large number of short-blocklength codewords in parallel, while it delivers incremental redundancy, without feedback, only to the decoders that need incremental redundancy.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/430,093 filed on Feb. 10, 2017, incorporated herein by reference inits entirety, which claims priority to, and the benefit of, U.S.provisional patent application Ser. No. 62/294,093 filed on Feb. 11,2016, incorporated herein by reference in its entirety.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

This invention was made with Government support under 1162501, awardedby the National Science Foundation. The Government has certain rights inthe invention.

INCORPORATION-BY-REFERENCE OF COMPUTER PROGRAM APPENDIX

Not Applicable

NOTICE OF MATERIAL SUBJECT TO COPYRIGHT PROTECTION

A portion of the material in this patent document is subject tocopyright protection under the copyright laws of the United States andof other countries. The owner of the copyright rights has no objectionto the facsimile reproduction by anyone of the patent document or thepatent disclosure, as it appears in the United States Patent andTrademark Office publicly available file or records, but otherwisereserves all copyright rights whatsoever. The copyright owner does nothereby waive any of its rights to have this patent document maintainedin secrecy, including without limitation its rights pursuant to 37C.F.R. § 1.14.

BACKGROUND 1. Technical Field

The technology of this disclosure pertains generally to high-throughputcommunications using error control codes, and more particularly to usingvariable-length codes with incremental redundancy to approach theShannon capacity without using feedback to control the transmission ofincremental redundancy.

2. Background Discussion

It is well-known that carefully designed error-control codes with verylong blocklengths can closely approach theoretical capacity. Forexample, as demonstrated in 2001 by Chung et al. (“On the design oflow-density parity-check codes within 0.0045 dB of the Shannon limit.IEEE Comm. Let., 5(2):58-60, February 2001”), a low-density parity-check(LDPC) code with a blocklength of 10⁷ bits can achieve a bit error rateof 10⁻⁶ at a signal-to-noise ratio (SNR) within 0.04 dB of the Shannonlimit. However, approaching capacity while simultaneously achieving highthroughputs on the order of 100 gigabits per second (Gbps), or more,remains an active area of research.

One example application where high throughput is critical is that ofoptical transport networks (OTNs). The OTU G.975.1 standard describesforward error correction (FEC) for high-bit-rate dense wavelengthdivision multiple-access (DWDMA) submarine systems. This standarddescribes “super FEC” schemes that have a higher error-correctioncapability than the (255,239) Reed-Solomon code, which is the baselinefor OTNs. Recent approaches for OTNs include Staircase codes and BraidedBCH Codes, which provide significant improvements over the “super FECs”at high throughputs on the order of 100 Gbps. Systems that approach thehard-decoding capacity have been proposed. Spatially-coupled (SC) LDPCcodes with windowed decoding provide a possible solution tohigh-throughput communication systems with soft decoding.

The growing demand for data drives a demand for improved performanceover difficult channels. This pushes implementations toward softdecoding and even higher throughputs. Systems constrained to harddecoding cannot approach the soft-decoding capacity. At high throughputsbeyond 100 Gbps, the complexity of place-and-route forcapacity-approaching schemes, such as iterative belief-propagationdecoding of a long-blocklength LDPC code, present a significantchallenge. Another factor affecting complexity for soft decoding is thatiterative belief-propagation decoders require a larger number ofiterations as their operating point approaches the Shannon limit. Athird concern is the ability to provide a guarantee on frame error rate(FER) that meets the requirements of high-throughput networks, whichsometimes require FERs below 10⁻¹⁵. Guaranteeing low FERs forlong-blocklength LDPC codes is difficult as the error-floor behavior ofLDPC codes is hard to characterize analytically. Even with windowedSC-LDPC codes, there are concerns about frame error rate guarantees.

High throughputs naturally allow the processing of a large amount ofdata, which provides the long blocklengths that allow capacity to beclosely approached. What is needed is a way to harvest the ergodicitybenefits of long blocklengths while somehow achieving the decodercomplexity of a short-blocklength code. Feedback allows shortblocklength codes to approach capacity, but such feedback is notpractical in a high-throughput system.

Accordingly, a need exists for error control mechanisms which operate athigh throughputs near capacity. The present disclosure fulfills thatneed and overcomes drawbacks to previous error control technologies.

BRIEF SUMMARY

In our prior work, it was demonstrated that capacity can be approachedwith short-blocklength convolutional codes and low-density parity-check(LDPC) codes that use simple ACK/NACK feedback controlling thetransmission of additional incremental redundancy. In the presentdisclosure, short-blocklength codes are utilized to approach capacitywith incremental redundancy, but without the need of feedback. A largenumber of short-blocklength codewords are transmitted and decoded inparallel. Incremental redundancy is delivered, without feedback, only tothe decoders that need it.

A key concept of this technology is to translate a capacity-approachingperformance of prior-art systems with feedback to a presently disclosedsystem that does not use feedback.

By way of example, and not of limitation, an embodiment of thetechnology described herein is a system that implements manycapacity-approaching short-blocklength encoders and decoders in parallelthat are coupled through common incremental redundancy. Source andchannel coding delivers incremental redundancy only to the decoders thatneed it without requiring feedback. The amount of redundancy needed canbe determined by central limit theorem arguments. Delivering thatredundancy only to the parallel decoders that need it is equivalent tosolving a specific problem in joint source-channel coding with sideinformation available only at the receiver. At the receiver, sideinformation comes from the parallel decoders that have alreadysuccessfully identified their codewords. These parallel decoders providethis side information to the incremental redundancy decoder, which usesit to provide incremental redundancy to the parallel decoders that havenot yet successfully identified their codewords. Several methods aredisclosed to achieve this, including the prior-art method of Zeineddineand Mansour, “Inter-frame coding for broadcast communication. IEEE J.Select. Areas Commun, vol. 34, no. 2, February 2016, pp. 437-452,” whichwas used to combat fading in a broadcast setting.

The disclosed technology uses many capacity-approachingshort-blocklength codes in parallel to create a long-blocklength systemwithout feedback. The performance of the technology of this disclosureis limited by that of the original short-blocklength code in a systemwith m rounds of non-active feedback.

Variable-length codes with average blocklengths of around 500 symbolscan, with non-active feedback, closely approach capacity in theory andin practice. Variable-length codes, whose blocklengths vary through theuse of incremental redundancy that is added only when needed, canachieve higher throughputs than fixed-blocklength codes for a specifiedaverage blocklength. Polyanskiy (Y. Polyanskiy, H. V. Poor, and S.Verd´u. “Channel coding rate in the finite blocklength regime.” IEEETrans. Inf. Theory, 56(5):2307-2359, May 2010.) has established thisanalytically and in our prior work, we have also demonstrated this inpractice with carefully designed variable-length codes and incrementalredundancy transmissions having carefully designed lengths, used withfeedback.

Thus, the present disclosure combines variable-length codes designed toapproach capacity with short blocklengths with the ability to compressand encode incremental redundancy into a common pool of redundancy toproduce a capacity-approaching system that does not require feedback butreaps many of the complexity benefits of short-blocklength codes.

Further aspects of the technology described herein will be brought outin the following portions of the specification, wherein the detaileddescription is for the purpose of fully disclosing preferred embodimentsof the technology without placing limitations thereon.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

The technology described herein will be more fully understood byreference to the following drawings which are for illustrative purposesonly:

FIG. 1 is a plot of prior art feedback performance approaching capacityat short blocklengths using feedback.

FIG. 2 is a block diagram of a decoder side utilizing numerous parallelindependent decoders with incremental redundancy provided as needed inlater stages, according to an embodiment of the present disclosure.

FIG. 3 is a plot of empirical complementary cumulative distributionfunction (c.d.f.) and the Gaussian approximation (Q-function)corresponding to the rate R_(S) at which the NB-LDPC code was able todecode successfully.

FIG. 4 is a plot of throughputs as a function of the number m ofincremental transmissions permitted.

FIG. 5 is a block diagram of encoded incremental redundancy with sideinformation from successful decoders, according to an embodiment of thepresent disclosure.

FIG. 6 is a Raptor-like protograph for a two-mode channel, according toan embodiment of the present disclosure.

FIG. 7 is a block diagram of a prior-art technique of inter-framecoding, showing variable-length code inputs and outputs, along withtransmitted symbols.

FIG. 8 is a plot of regular and irregular LDGM code performance curvesin addition to performance predicted by density evolution for codesdesigned for an embodiment of the present disclosure.

DETAILED DESCRIPTION

1. Introduction

This disclosure describes a new system to harvest the ergodicitybenefits of long blocklengths while achieving the decoder complexity ofa short-blocklength code by leveraging the performance ofshort-blocklength codes with incremental redundancy. This system canapproach capacity at high throughputs while permitting strong guaranteeson frame error rate (FER) performance.

Prior work considers both “active” and “non-active” feedback systems.Non-active feedback only tells the transmitter when to stop. At specificintervals in a feedback system, the receiver sends a message to thetransmitter (an ACK or NACK) indicating whether or not more incrementalredundancy is needed. This specification uses the parameter m toindicate the maximum number of ACK/NACK messages that can be sent beforethe system must give up on a transmission. After m transmissions, if themessage cannot be decoded, it is considered an error.

Active feedback goes beyond indicating whether more redundancy is neededand instructs the transmitter about what information would be mostuseful for decoding. Active feedback, and especially its most extremeform of active sequential hypothesis testing, can dramatically improveperformance at extremely short blocklengths, performing above the randomcoding bound, but at the cost of extreme complexity. However, importantfor the present disclosure, even non-active feedback systems canapproach capacity for slightly longer average blocklengths. Non-activefeedback systems can approach capacity with blocklengths on the order of500 symbols. The present disclosure focuses on translating a non-activefeedback system to a system that has no feedback.

FIG. 1 describes prior-art feedback performance for the example of abinary-input (BI) additive white Gaussian noise (AWGN) channel withframe error rate (FER) required to be less than 10⁻³. For a systemtransmitting k symbols at an average blocklength of λ, the throughputR_(t) is defined by R_(t)=k/λ. For reference, FIG. 1 shows the curves ofpossible throughput R_(t) as a function of λ for some values of k. Theperformance characterization for fixed-blocklength codes is from thepreviously described Polyanskiy article, and is based on the normalapproximation, which is shown in that reference to be accurate forblocklengths as small as 100 symbols.

The computation of the random coding lower bound on the performance ofvariable-length codes with feedback is based on the analysis fromreference “Y. Polyanskiy, H. V. Poor, and S. Verd´u. Feedback in thenon-asymptotic regime. IEEE Trans. Inf. Theory, 57(8):4903-4925, August2011.”

In FIG. 1, curves are shown with respect to the following references:

(a) “Y. Polyanskiy, H. V. Poor, and S. Verd´u. Feedback in thenon-asymptotic regime. IEEE Trans. Inf. Theory, 57(8):4903-4925, August2011” This is reflected in the “Random-coding lower bound” line of thelegend of FIG. 1.

(b) “Y. Polyanskiy, H. V. Poor, and S. Verd´u. Channel coding rate inthe finite blocklength regime. IEEE Trans. Inf. Theory, 56(5):2307-2359,May 2010.” This is reflected in the “Fixed blocklength no feedback” lineof the legend of FIG. 1.

(c) “A. R. Williamson, T.-Y. Chen, and R. D. Wesel. Firing the genie:Two-phase short-blocklength convolutional coding with feedback. In IEEEInf. Theory and Application. Workshop, pages 1-6, San Diego, Calif.,February 2013.” This is reflected in the “1024 CC” line of the legend ofFIG. 1.

(d) “A. R. Williamson, T.-Y. Chen, and R. D. Wesel. Variable-lengthconvolutional coding for short blocklengths with decision feedback. IEEETrans. Commun., 63(7):2389-2403, July 2015.” This is reflected in the“64 TBCC” line of the legend of FIG. 1.

(e) “K. Vakilinia, A. R. Williamson, S. V. S. Raganathan, D. Divsalar,and R. D. Wesel. Feedback systems using non-binary LDPC codes with alimited number of transmissions. In IEEE Information Theory Workshop,pages 167-171, Hobart, Tasmania, Australia, November 2014.” This isreflected in the “NB LDPC” lines of the legend of FIG. 1.

The curves of FIG. 1 show systems with feedback that closely approachthe performance promised by the Polyanskiy reference listed earlier for“Feedback in the non-asymptotic regime” in the range of averageblocklengths below 500 bits. For values of k=16, k=32, k=64, and k =89these throughput results exceed Polyanskiy's random coding lower bound.As the average blocklength becomes larger, the random coding lower boundis more predictive.

The key point of FIG. 1 is that variable-length codes with feedback canapproach capacity at very short blocklengths. In FIG. 1, therandom-coding lower bound for a system with feedback is 0.27 dB from theShannon limit for k=280 with a blocklength of less than 500 bits.Looking at implemented codes for k=280 in FIG. 1, the m=∞ non-binaryLDPC (NB-LDPC) code is 0.53 dB from the Shannon limit. More importantlyfor the present disclosure, the NB-LDPC non-active feedback system inFIG. 1 that uses ten rounds of single-bit feedback still operates within0.65 dB of the Shannon limit with an average blocklength of less than500 bits.

Expressed another way, the m=10 non-binary LDPC (NB-LDPC) code achievesan R_(t) value that is 93% of the Shannon capacity of the Bi-AWGNchannel. The m=5 TBCC system achieves an R_(t) value that is 82% of theShannon capacity of the Bi-AWGN channel. From this it is seen thatpreviously developed systems demonstrate that capacity can be approachedwith short-blocklength convolutional codes and LDPC codes that usesimple ACK/NACK feedback controlling the transmission of additionalincremental redundancy.

The present disclosure builds on prior approaches by usingshort-blocklength codes to achieve capacity with incremental redundancy,but departs from those approaches by not using feedback. In our“feedback-less system”, a large number of short-blocklength codewordsare transmitted and decoded in parallel. Incremental redundancy isdelivered without feedback but still only to the decoders that need it.

As a simple example, consider two decoders that are operating inparallel with feedback. Assume that the transmitter knows statisticallythat one decoder will succeed in decoding and one will send feedbackrequesting the next segment of incremental redundancy. Instead of havingthe receiver send feedback, the transmitter can send the XOR of the twoincremental redundancy transmissions, and the decoder that succeeds canremove its incremental part from the XOR so that incremental redundancyis available for the parallel decoder that needs it.

To further understand this feedback-less technology, considerincremental redundancy for a rate-compatible code delivered in a smallnumber of segments of possibly varying length. The detailed embodimentsbelow describe how minimal throughput is lost by restricting theredundancy to be delivered in a small number of segments, if the lengthsof the segments are optimized using a previously-developed techniquecalled sequential differential approximation. Embodiments of the presenttechnology use many such codes in parallel. Each of the small number ofsegments has a corresponding decoding stage. To send the appropriateamount of redundancy for the next stage, the transmitter needs to knowthe minimum number of parallel decoders that will have succeeded in theprevious stages. This can be estimated using the law of large numbers.It should be noted that the transmitter according to the presentdisclosure does not need to know (and cannot know without feedback)which of the parallel decoders have succeeded at each stage. Because thetransmitter does not know which of the parallel decoders have succeeded,efficient communication requires that the transmitter encode across theincremental redundancy for all of the parallel decoders to produce a(compressed) transmission that is of the appropriate length for thatstage. The incremental-redundancy decoder must have the ability toprocess the encoded incremental redundancy it receives from thetransmitter, in order to produce soft reliability values for theincremental redundancy bits needed by the parallel decoders that havenot yet successfully decoded.

According to an embodiment of our feedback-less technology, the amountsof incremental redundancy delivered at each stage can be carefullydesigned by utilizing central limit theorem (CLT) arguments that capturethe probability of a certain number of decoders succeeding at eachstage. Then, the present disclosure makes use of one of several possibletechniques to deliver incremental redundancy at each stage only to theshort-blocklength decoders that have not yet successfully decoded.

As described in embodiments below, one technique treats this specializeddelivery of incremental redundancy as a code-design problem for a noisychannel that has independent and randomly varying side information thatis only available at the receiver. This side information is produced bythe short-blocklength decoders that have succeeded in earlier stages.Some of the parallel short-blocklength decoders may fail even after thefinal stage, and embodiments of the present disclosure utilize apacket-level erasure code to provide strong FER guarantees in thecontext of the well-understood behavior of the parallel decoders withincremental redundancy.

In the disclosed feedback-less technology, the many parallelshort-blocklength codes are connected by the incremental-redundancyencoder and by a packet-level erasure code, both of which code acrossall the parallel codes. Thus, in the end, this is a fixed-to-fixed,long-blocklength system, and it is not surprising that it can approachcapacity. What is special is that the decomposition allows capacity tobe approached with low-complexity short-blocklength decoders and massiveparallelism. The capacity-approaching ergodicity benefits of along-blocklength code are obtained while still essentially usingshort-blocklength decoders.

2. Detailed Embodiments

The following describes embodiments of the technology, which are systemsthat approach capacity but that do not require any feedback but useshort-blocklength codes that would work well in the context of anon-active feedback system. In particular, embodiments will use shortblocklength codes that are variable-length in that they have m stages ofincremental redundancy, I₁, I₂, . . . , I_(m), where m≥2. Embodimentsuse L such short-blocklength codes in parallel. It should be appreciatedthat some embodiments of the technology will have some of these L shortblocklength codes be redundant in the sense that they are part of afixed-rate packet-level erasure code that will allow successful decodingeven when some of the short-blocklength codes cannot be decoded.

2.1 Encoding

The transmitter sends the initial transmission I₁ ⁽¹⁾, I₁ ⁽²⁾, . . . ,I₁ ^((L)) for each of the L short-blocklength codes directly over thechannel n the usual way. However, for the second stage and any laterstages, the incremental redundancy associated with the Lshort-blocklength codes is combined and compressed to account for thefact that not all of the incremental redundancy is needed at thereceiver because some of the L short-blocklength codes will havesuccessfully decoded using only the earlier stages of incrementalredundancy.

Thus, after the first stage, the transmitter sends symbols that are theresult of an incremental redundancy encoder. For example, in the secondstage of transmission the encoded incremental redundancy F₂ (I₂ ⁽¹⁾, I₂⁽²⁾, . . . , I₂ ^((L))) is transmitted over the channel, where theencoder F₂ (●) combines and compresses the incremental redundancy of thesecond stage of the short-blocklength codes. Thus, the full transmissionof m stages of incremental redundancy for all L short-blocklengthcodewords, which we refer to as a super-frame, is made up of I₁ ⁽¹⁾, I₁⁽²⁾, . . . , I₁ ^((L)) and F₂ (I₂ ⁽¹⁾, I₂ ⁽²⁾,

, I₂ ^((L))),

, F_(m)(I_(m) ⁽¹⁾, I_(m) ⁽²⁾,

, I_(m) ^((L))).

It should be appreciated that the amount of information transmitted foreach stage and thus the number of symbols transmitted for each stageneed not be the same. In fact, it is expected that the number of symbolstransmitted for each stage will be different both because the amount ofincremental redundancy required by a single short-blocklength code willtypically be different for different stages and because the number ofshort blocklength codes that require a stage of incremental redundancydecreases with each stage as more decoders successfully decode. However,the amount of information transmitted for each stage and thus for thesuperframe as a whole is a fixed value. It should be appreciated thatthe present disclosure is not a “rateless” code that provides acontinuous stream of redundancy.

According to our feedback-less technology, the amount of informationcommunicated in each stage of incremental redundancy is determined priorto transmission is implemented by the incremental redundancy encoder.This is in contrast to other systems which propose a “rateless” systemthat sends a continuous stream of random linear combinations untildecoding is successful. Another difference between our disclosedfeedback-less technology and these “rateless” systems is that thepresent disclosure uses a fixed-rate packet-level erasure code to allowdecoding when some of the short blocklength codes cannot be decoded.Rateless systems explicitly do not use any packet-level erasure codingin conjunction with their systems; and a packet-level erasure code isnot needed because the stream of random linear combinations continuesindefinitely until decoding is successful.

2.2 Decoding

FIG. 2 illustrates an example embodiment 10 of a first stage of an m-stage decoder for the presented feedback-less technology. Dashed linesrepresent communication paths that are not utilized in this example butare potential paths for different cases when different sets of theparallel decoders succeed. Transmission begins by sending the firstincrements 12 I₁ ⁽¹⁾, I₁ ⁽²⁾, I₁ ^((L)) (26 a, 26 b, . . . , 26L)through the channel, for example an AWGN channel 14 is depicted, coupledto the L parallel decoders 16. In addition F₂ (●) 28 is shown as F₂(I₁⁽¹⁾, I₁ ⁽²⁾, . . . , I₁ ^((L))) coupled to channel 14. It will beappreciated that each row of elements shown here will be of length L,where L is chosen large enough that ergodicity causes the fraction ofcodes decoding to be well-behaved. Values of L such as 10,000 or even100,000 could be used, but values as small as 1000 or 100 might be usedin some systems. It will be appreciated that smaller values of L willrequire more powerful erasure coding and decrease throughput.

All L decoders 16 (30 a, 30 b, . . . , 30L) perform the first stage ofparallel decoding. Some of these parallel decoders will achievesuccessful decoding in the first stage, which is recognized by, forexample a CRC check passing stage 18, shown with checks 32 a, 32 bthrough 32L. This is shown in the figure for the first parallel decoder,which is able to successfully decode using the initial transmission I₁⁽¹⁾. If the CRC check passes, then the incremental redundancy decoder 22is utilized, otherwise a second decoder stage 20 is utilized, as shownwith decoders 34 a, 34 b, . . . , 34L. Output from the Incrementalredundancy decoder is coupled to the second stage decoders. Output 24 isshown from the second stage decoders.

For these codewords for which the CRC check passes, the later stages ofdecoding are not needed and are not performed. The successful paralleldecoders can use the decoded message to compute I₂ ⁽¹⁾ and provide it asside information to the incremental-redundancy decoder. Thus, theincremental-redundancy decoder has access to some of the message bitsthat were the inputs to the encoder F₂ (●). This is shown in the figurewhere the first decoder provides I₂ ^((i)) to the incremental-redundancydecoder.

It should be appreciated that for the overall system to accomplish itsgoal, F₂ (●) performs both source and channel coding functions becauseit is compressing the set of increments I₂ ⁽¹⁾, I₂ ⁽²⁾, . . . , I^((L))based on the amount of side information that will be available at thereceiver and it is preparing the compressed information for transmissionover the channel. However, in order to achieve the overall system goalof replicating the performance of the short-blocklength codes used withfeedback, it is not necessary that the incremental-redundancy decoderrecover the increments I₂ ^((i)) perfectly. Rather, what is needed is todeliver to the second-stage decoders reliability information about theincrements I₂ ^((i)) that is similar (in quality) to the reliabilityinformation that would have been received if these increments had beentransmitted directly over the channel.

The incremental-redundancy decoder, which will be discussed at lengthbelow, uses the encoded incremental redundancy F₂ (I₂ ⁽¹⁾, I₂ ⁽²⁾, . . ., I₂ ^((L))) transmitted over the channel and the side informationprovided by the successful decoders to provide reliabilities for I₂^((i)) to the decoders that were not successful in the first stage. Thisis shown in the figure where the incremental-redundancy decoder providesthe reliabilities r(I₂ ⁽²⁾) and r(I₂ ^((L))) to the second and Lthdecoders, respectively. In the second stage, for example, the seconddecoder can now attempt decoding based on both reliabilities from theinitial transmission r(I₁ ⁽²⁾) and from the incremental redundancy r(I₂⁽²⁾). This process continues for m stages. Even after all m stages ofdecoding are completed, some decoders may still not have succeeded.

Although not depicted in FIG. 2, it is important to understand that inthe overall system, it is the final packet-level erasure decoding stepthat will recover the super-frame despite the failure of a small numberof the decoders even after the final stage. Some embodiments might besatisfied with the failure of a small number of the parallel decodersand forego the packet-level erasure decoding.

2.3 Determining How Many Decoders will Succeed at Each Stage

To manage the flow of incremental redundancy (i.e., to determine howmuch information should be transmitted at each stage) without feedback,the present disclosure makes use of the power of the Gaussianapproximation introduced on the rate that a channel can support atfinite blocklength.

According to a Gaussian approximation, information density i(X,Y) isdefined as

$\begin{matrix}{{i\left( {X,Y} \right)} = {\log_{2}{\frac{f_{Y|X}\left( y \middle| x \right)}{f_{Y}(y)}.}}} & (1)\end{matrix}$

The expected value of i(X,Y) is the capacity of the channel. For theexample of a BI-AWGN channel with noise z_(k), i(X,Y)=1−log₂ (1+e^(−2(z)^(k) ^(+1)σ) ² )=i (z_(k)). The accumulated information density I_(S),at the receiver at time N_(S) of successful decoding is

$\begin{matrix}{I_{s} = {\sum\limits_{k = 1}^{N_{s}}{{i\left( z_{k} \right)}.}}} & (2)\end{matrix}$

It will be noted that Eq. (2) is a sum of independent random variablesfor which the central limit theorem converges quickly to a normaldistribution, leading to the normal approximation. An importantconsideration for the present disclosure is whether the rate at which apractical decoder succeeds also follows such a normal distribution.

FIG. 3 depicts complementary cumulative distribution. Althoughoriginally computed for a system with feedback, the plot indicates thatthe answer to this question is in the affirmative. The figure shows thatfor a short-blocklength NB-LDPC code used by K. Vakilinia, A. R.Williamson, S. V. S. Raganathan, D. Divsalar, and R. D. Wesel in“Feedback systems using non-binary LDPC codes with a limited number oftransmissions”. In IEEE Information Theory Workshop, pages 167-171,Hobart, Tasmania, Australia, November 2014; the empirical complementarycumulative distribution function on the rate at which decoding issuccessful is very closely approximated by a Gaussian distribution forthis example of the BI-AWGN channel with SNR of 2 dB. Similarly,accuracy of the Gaussian approximation to predict the rate at whichdecoding is successful has been confirmed for similar NB-LDPC codes inhigher-SNR AWGN channels that require larger constellations and infading channels with channel state information known at the receiver.

In FIG. 3 one can see that R_(S) is well-approximated by a Gaussian withmean μ_(S)=E[R_(S)] and variance σ_(S) ²=Var(R_(S)):

$\begin{matrix}{{f_{R_{S}}(r)} = {\frac{1}{\sqrt{2{\pi\sigma}_{S}^{2}}}{e^{- \frac{{({r - \mu_{S}})}^{2}}{2\sigma_{S}^{2}}}.}}} & (3)\end{matrix}$

The accuracy of the normal approximation allows the prediction atspecified blocklengths of how many decoders have succeeded and how manywill need further incremental redundancy. The cumulative distributionfunction (c.d.f.) of N_(S), the blocklength at which decoding issuccessful, is F_(N) _(S) (n)=P(N_(S)≤n)=1−F_(R) _(S) (k/n). Taking thederivative of F_(N) _(S) using the Gaussian approximate on of F_(R) _(S)produces the following “reciprocal-Gaussian” approximation for theprobability density function (p.d.f.) of N_(S):

$\begin{matrix}{{f_{N_{S}}(n)} = {\frac{k}{n^{2}\sqrt{2{\pi\sigma}_{S}^{2}}}{e^{- \frac{{({{k/n} - \mu_{S}})}^{2}}{2\sigma_{S}^{2}}}.}}} & (4)\end{matrix}$

For incremental redundancy to be employed without feedback, the presentdisclosure must break the communication into a number of distincttransmissions (stages) of incremental redundancy. In order to determinehow much information to transmit for each stage, the transmitter needsto know what fraction of the parallel short-blocklength decoders willneed incremental redundancy at each stage because they have not yetdecoded successfully.

The number of incremental transmissions is limited to m, with theincrements {I₁, I₂, . . . , I_(m)} being introduced in a previoussection. It should be noted that the cumulative blocklength N_(j) at thejth stage is simply the sum of the first j increment lengths. Using thep.d.f. of N_(S) from Eq. (4), ideas are used from the articles abovefrom Vakilinia, Williamson, Raganathan, Divsalar, and Wesel, which wereoriginally intended for feedback systems to compute the probability thateach decoder will require a particular incremental transmission at aparticular stage even when feedback is not available. For N_(j)<N_(j+1),the probability of a successful decoding attempt at blocklength N_(j+1)but not at N_(j) is

$\begin{matrix}{{\int_{N_{n}}^{N_{j + 1}}{{f_{N_{S}}(n)}{dn}}} = {{\int_{N_{n}}^{N_{j + 1}}{\frac{k}{n^{2}\sqrt{2{\pi\sigma}_{S}^{2}}}e^{- \frac{{({{k/n} - \mu_{S}})}^{2}}{2\sigma_{S}^{2}}}{dn}}} = {{Q\left( \frac{{k/N_{j + 1}} - \mu_{S}}{\sigma_{S}} \right)} - {Q\left( \frac{{k/N_{j}} - \mu_{S}}{\sigma_{S}} \right)}}}} & (5)\end{matrix}$

The disclosed feedback-less technology combines the ability toaccurately compute this probability using the approximation of Eq. (5)or, without deviating from the scope of the present disclosure, adifferent approximation with the use of the ergodicity of many paralleldecoders to manage the flow of incremental redundancy. This is a coreconcept of this technology.

2.4 Optimizing the Blocklengths {N₁, N₂, . . . , N_(m)}

Separately, the tight Gaussian approximation discussed abovefacilitates, as described by the above article of Vakilinia, Williamson,Raganathan, Divsalar, and Wesel, which was originally for feedbacksystems, the optimization of the sequence of blocklengths {N₁, N₂, . . ., N_(m)} to maximize the throughput. The throughput is defined asR_(t)=E[K]/E[N], where E[N] represents the expected number of channelsused and E[K] is the effective number of information bits transferredcorrectly over the channel. The expression for E[N] is

$\begin{matrix}{{E\lbrack N\rbrack} = {{N_{1}{Q\left( \frac{{k/N_{1}} - \mu_{S}}{\sigma_{S}} \right)}} + {\sum\limits_{j = 2}^{m}{N_{j}\left\lbrack {{Q\left( \frac{{k/N_{j}} - \mu_{S}}{\sigma_{S}} \right)} - {Q\left( \frac{{k/N_{j - 1}} - \mu_{S}}{\sigma_{S}} \right)}} \right\rbrack}} + {N_{m}\left\lbrack {1 - {Q\left( \frac{{k/N_{m}} - \mu_{S}}{\sigma_{S}} \right)}} \right\rbrack}}} & (6)\end{matrix}$

The first term shows the contribution to the expected blocklength fromsuccessful decoding on the first attempt. The term

$Q\left( \frac{{k/N_{1}} - \mu_{S}}{\sigma_{S}} \right)$

is the probability of decoding successfully with the initial block ofN₁. Similarly, the terms in the summation are the contributions to theexpected blocklength from decoding that is first successful at totalblocklength N_(j) (at the jth decoding attempt). Finally, thecontribution to expected blocklength from not being able to decode evenat N_(m) is

$1 - {{Q\left( \frac{{k/N_{m}} - \mu_{S}}{\sigma_{S}} \right)}.}$

Even when the decoding has not been successful at N_(m), the channel hasbeen used for N_(m) channel symbols. The expected number of successfullytransferred information bits E[K] is

$\begin{matrix}{{{E\lbrack K\rbrack} = {{kQ}\left( \frac{{k/N_{m}} - \mu_{S}}{\sigma_{S}} \right)}},} & (7)\end{matrix}$

where

$Q\left( \frac{{k/N_{m}} - \mu_{S}}{\sigma_{S}} \right)$

is the probability of successfully decoding. It should be appreciatedthat E[K] depends only upon N_(m). In fact, E[K]≈k and is not sensitiveto specific choice of N_(m) for reasonably large values of N_(m).

The initial blocklength is N₁ and we seek the optimal blocklengths {N₁,N₂, . . . , N_(m)} to maximize the throughput. Over a range of possibleN₁ values, a technique we call sequential differential optimization(SDO), introduced in the previously cited Vakilinia, Williamson,Raganathan, Divsalar, and Wesel article as sequential differentialapproximation in the context of systems using feedback, selects {N₁, N₂,. . . , N_(m)} to minimize E[N] for each fixed value of N₁ by settingderivatives to zero as follows:

$\begin{matrix}\left\{ {N_{1},N_{2},\ldots \mspace{14mu},{{N_{m}\text{:}\mspace{14mu} \frac{\partial{E\lbrack N\rbrack}}{\partial N_{j}}} = 0},{{\forall j} = 1},\ldots \mspace{14mu},{m - 1}} \right\} & (8)\end{matrix}$

The principle of sequential differential optimization is that for eachj∈{2, . . . , m} the optimal value of N_(j) is found by setting

$\frac{\partial{E\lbrack N\rbrack}}{\partial N_{j - 1}} = 0$

yielding a sequence of relatively simple computations. This can beapplied to any p.d.f. for N_(S), and the p.d.f. of Eq. (4) should beconsidered as one example of the principle. The disclosed techniqueselects an integer value of N_(j) that approximates the real-numberedvalue of N_(j) that makes the previous choice of N_(j−1) optimal inretrospect.

For j=2,

$\frac{\partial{E\lbrack N\rbrack}}{\partial N_{j - 1}}$

depends only on {N_(j−1)=N₁, N_(j)=N₂} as follows:

$\frac{\partial{E\lbrack N\rbrack}}{\partial N_{1}} = {{Q\left( \frac{{\frac{k}{N_{1}} -}\mu_{S}}{\sigma_{S}} \right)} + {\left( {N_{1} - N_{2}} \right){Q^{\prime}\left( \frac{{\frac{k}{N_{1}} -}\mu_{S}}{\sigma_{S}} \right)}}}$

Thus, the system then solves for N₂ as

$N_{2} = \frac{{Q\left( \frac{{\frac{k}{N_{1}} -}\mu_{S}}{\sigma_{S}} \right)} + {N_{1}{Q^{\prime}\left( \frac{{\frac{k}{N_{1}} -}\mu_{S}}{\sigma_{S}} \right)}}}{Q^{\prime}\left( \frac{{\frac{k}{N_{1}} -}\mu_{S}}{\sigma_{S}} \right)}$

For j>2,

$\frac{\partial{E\lbrack N\rbrack}}{\partial N_{j - 1}}$

depends only on {N_(j−2,) N_(j−1,) N_(j,)} as follows:

$\begin{matrix}{\frac{\partial{E\lbrack N\rbrack}}{\partial N_{j - 1}} = {{Q\left( \frac{{k/N_{j - 1}} - \mu}{\sigma} \right)} + {\left( {N_{j - 1} - N_{j}} \right){Q^{\prime}\left( \frac{{k/N_{j - 1}} - \mu}{\sigma} \right)}} - {{Q\left( \frac{{k/N_{j - 2}} - \mu}{\sigma} \right)}.}}} & (9)\end{matrix}$

Thus, the system then solves for N_(j) as

$\begin{matrix}{N_{j} = {\frac{{Q\left( \frac{{k/N_{j - 1}} - \mu}{\sigma} \right)} + {N_{j - 1}{Q^{\prime}\left( \frac{{k/N_{j - 1}} - \mu}{\sigma} \right)}} - {Q\left( \frac{{k/N_{j - 2}} - \mu}{\sigma} \right)}}{Q^{\prime}\left( \frac{{k/N_{j - 1}} - \mu}{\sigma} \right)}.}} & (10)\end{matrix}$

For each possible value of N₁, SDO can be used to produce an infinitesequence of N_(j) values that solve Eq. (8). It should be appreciatedthat the SDO equations produce real numbers while the actual sequencelengths must be integers, so the values are rounded or alternativelyhandled, and by considering the floor and ceiling at each step of theoptimization, the tree of possible integer values can be explored.

The resulting sequence is an optimal sequence of increment lengths for agiven density of points in time, where each point is a decoding attempton the axis of transmission time. As N₁ increases, the density ofdecoding attempts decreases, lowering system complexity. Using SDO tocompute the optimal m decoding points is equivalent to selecting themost dense SDO-optimal sequence that when truncated to m points stillmeets the frame error rate target.

FIG. 4 depicts resulting throughputs obtained by using SDO to find theoptimal increment lengths for values of m in the range of 2≤m≤20 for thetarget FER of 10⁻³ for the NB-LDPC code from the previously citedVakilinia, Williamson, Raganathan, Divsalar, and Wesel article for k=96message bits. The figure illustrates that with m=10 decoding points, asystem can closely approach the performance of a system that has m=∞,which is a system that attempts decoding after every received symbol.

2.5 Performance Guarantees for Parallel Decoders

To provide shared incremental redundancy to the multiple independentshort-blocklength decoders, it is preferable that enough overallredundancy is provided to the system at each stage. Sequentialdifferential optimization (SDO) can specify the sizes of the incrementalredundancy transmissions. Additionally, for each incremental-redundancystage, there is an associated probability of successful decoding basedon the Gaussian approximation from Eq. (5), which is quite accurate.Again, other p.d.f.s for N_(S) could be used without deviating from thescope of the present disclosure. This probability leads directly to theexpected value of the number of frames that decode correctly at thatstage.

The embodied transmitter of the disclosure also preferably considers thevariation around that expected value. The overall approach of thisdisclosure is related to product coding, and to motivate this approachwe follow the example of Elias' early work on product codes (P. Elias.Error-free coding. Transactions of the IRE Professional Group onInformation Theory, 4(4):29-37, 1954.) and use arguments based on thebinomial cumulative distribution. Suppose that a super-frame is composedof L=1000 frames (1000 short-blocklength codes) and the expected numberof successfully decoded frames after the initial transmission I₁ is 200out of 1000. Define the length of the incremental redundancytransmission I₂ to be

(I₂) bits. At least 800

(I₂) bits need to be received to allow the remaining decoders to proceedto the next stage. However, there is a significant probability thatfewer than 200 decoders will succeed. In order to meet an overall frameerror probability of 10⁻¹⁵ with ten stages of redundancy, each stageneeds to have a failure rate of less than 10⁻¹⁶. Noting that thecumulative of the binomial with 1000 trials and probability of stage-1success P (i∈S₁)=0.2 is equal to 9.6×10⁻¹⁷ for 103 or fewer successes,incremental redundancy needs to be provided for 1000−103=896 of the 1000frames. A similar analysis indicates that in a later stage when theexpected total number of successes is 800, incremental redundancy shouldbe provided for 310 decoders since the cumulative of the binomial with1000 trials and p=0.8 is equal to 7.1×10⁻¹⁷ for 689 or fewer successes.Thus, there is some overhead (in additional incremental redundancy)associated with not knowing exactly how many decoders need additionalincremental redundancy.

However, it is important to note that this additional redundancy isrequired by only about a tenth of the decoders and that most of theblocklength is in the initial transmission I₁; therefore, this overheadrepresents a relatively small fraction (typically below 3%) of theoverall transmission length.

Consider the example of using the NB-LDPC with feedback code for k=280whose performance was shown in FIG. 1. Table 1 provides the incrementsI_(j) computed by SDO, the cumulative blocklengths N_(j), and theprobability of successful decoding at each stage. Using theseparameters, the binomial analysis described above results in an overheadof 2.6% for 1000 decoders in parallel. The initial transmission of 428bits, which does not contribute to this overhead, is much larger thanthe subsequent incremental transmissions; which aids in keeping theoverhead small. Of course, as more parallel decoders are employed thedistribution concentrates further. For example, if 10,000 decoders areutilized in parallel then the overhead drops to 0.8%. Relaxing the FERrequirement also reduces overhead. For an overall FER of 10⁻⁶, requiringeach stage to have FER 10⁻⁷, the overhead for 1000 parallel decoders is1.6%.

The discussion above only ensures that the incremental redundancy wouldbe provided to each short-blocklength code as if it were providingfeedback to the transmitter. The short-blocklength codes shown in FIG. 1for the NB-LDPC code with m=10 achieve a modest FER of 10⁻³ with anaverage blocklength of 500 bits. However, the present disclosure seeksframe error rates well below 10⁻³. Including a packet-level erasure codecan allow extremely low superframe FERs even with a frame FER of 10⁻³.As an example, for a 1000-frame super-frame with each individual framefailing with a probability of 10⁻³, an erasure code that can correctsixteen erased frames yields a super-frame failure rate of 1.0×10⁻¹⁵,obtained by subtracting the binomial cumulative distribution functionfor sixteen or fewer failures in 1000 trials with a failure probabilityof 10⁻³.

Because of the CRC in each frame, failures will appear as erasures atthe receiver. Relatively short CRCs can still ensure that the undetectederror probability is well below 10⁻¹⁶ by incorporating, for example, thecodeword structure into the design of the CRC as described by C. Y. Lou,B. Daneshrad, and R. D. Wesel in “Convolutional-code-specific CRC codedesign. IEEE Trans. Commun., 63(10):3459-3470, October 2015. Thesimulation results in FIG. 1 for the m=10 NB-LDPC system operating 0.65dB from capacity already includes the overhead of an 8-bit CRC used toidentify frames that require additional redundancy.

It should be appreciated that the CRC check is only one method fordetermining that a short-blocklength frame has been received. Othertechniques for determining that a frame has been successfully receivedcan be employed and still not deviate from the scope of the presentdisclosure. One such example is the reliability-output Viterbi algorithmthat was employed by Williamson in the previously described article“Variable-length convolutional coding for short blocklengths withdecision feedback”.

2.6 Utilizing Successful Decoder Data

One aspect of the present disclosure is a system that deliversincremental redundancy to the parallel decoders that need it withoutwasting transmission symbols providing redundancy to parallel decodersthat have already succeeded. Using, for example, the Gaussianapproximation, the amount of incremental redundancy required at eachstage can be established as discussed above. The following discusses howto ensure that the incremental redundancy can be accessed by exactly thedecoders that need it.

FIG. 5 illustrates an embodiment 50 showing how information about theneeded incremental redundancy is provided to the incremental redundancydecoder. Incremental redundancies 54 are seen at input 52 of thedecoder, shown with parallel decoders 56, with X_(n) output 60, summedwith which is summed with Z_(n) 64 at junction 66, to outputY_(n)=X_(n)+Z_(n) 68. Side information from successful decoders is seenprocessed 58 and outputting {I_(j) ^((i)):i∈S_(j−1)}62. The transmitterencodes the incremental redundancies of stages j≥2 for each of theparallel decoders with the encoding functions F_(j)(I_(j) ⁽¹⁾, I_(j)⁽²⁾, . . . I_(j) ^((L))) that produce the sequence of transmissionsymbols {x_(n)}. Upon decoding at (the previous) stage j−1, the Lparallel decoders divide into two sets, the successful decodersS_(j−1)⊂{1, . . . , L} and the unsuccessful decoders U_(j−1)⊂{1,. . . ,L}. The incremental-redundancy decoder uses {y_(n)}, which is the noisyversion of the transmitted sequence {x_(n)}, and the side information of{I_(j) ⁽¹⁾: i∈S_(j−1)} to produce reliability values for {I_(j)⁽¹⁾:i∈U_(j−1)}. As discussed above, the number of successful decoders(decoders in S_(j)) increases with j so that more side information isrevealed. Thus, the function F_(j)(●) can send proportionally fewersymbols x_(n). As discussed above, the number of decoders that need tohave incremental redundancy at each stage can be accurately predicted toensure a specified target super-frame error rate. The following nowdescribes the structure of the encoder F_(j)(I_(j) ⁽¹⁾, I_(j) ⁽²⁾, . . ., I_(j) ^((L))) and how its symbols, together with the side information{I_(j) ⁽¹⁾:i∈S_(j−1)}, are used to produce the needed reliabilityvalues.

2.7 Lossy Source Coding Approach to Incremental Redundancy Encoder

The goal of the incremental-redundancy decoder is to providereliabilities to the parallel decoders that have not yet successfullydecoded. Because the bits of the increments {I_(j) ⁽¹⁾: i∈U_(j−1)} donot need to be recovered exactly, this is essentially a lossy codingproblem. For this set-up of lossy transmission of a source over achannel with side information about the source, source coding (with sideinformation) followed by channel coding has been shown to be optimal(e.g., article S. Shamai, S. Verdu, and R. Zamir. Systematic LossySource/Channel Coding. IEEE Trans. Inf. Theory, 44(2): 564-579, March1998). Hence, using the separation of source and channel coding toneglect for a moment the noisy channel y=x +z, consider the problem oflossy source coding of I_(j) ⁽¹⁾, I_(j) ⁽²⁾

, I_(j) ^((L)) with side information {I_(j) ⁽¹⁾:i∈S_(j−1)}. Let thesource X be the bits of {I_(j) ⁽¹⁾, I_(j) ⁽²⁾,

, I_(j) ^((L))} and the side information Y being the result of passing Xthrough an erasure channel with erasure probability p equal to theprobability that a decoder is unsuccessful at stage j−1.

The rate distortion function for source coding with side information atthe decoder was established in general by Wyner and Ziv in article “TheRate-Distortion Function for Source Coding with Side Information at theDecoder. IEEE Trans. Inf. Theory, 22(1):1-10, January 1976.” Verdu andWeissman and Perron et al. (S. Verdu and T. Weissman. “The InformationLost in Erasures.” IEEE Trans. Inf. Theory, 54(11):5030-5058, November2008, and E. Perron, S. Diggavi, and I. E. Telatar. “Lossy Source Codingwith Gaussian or Erased Side-Information”. In Proc. IEEE Int. Symp.Inform. Theory, pages 1035-1039, June 2009.) considered the specificcase of side information Y that is an erased version of the source X.For our research, the key result from Verdu and Weissman is Theorem 18(see also Theorem 1 of the Perron and Diggavi article), which considersany source X taking values in a discrete set X with distortion measured:X×X→R⁺ and side information Y being an erased version of X witherasure probability p. The rate distortion function with thiserasure-channel side information is

R _(X|Y)(D)=pR _(X)(D/p)   (11)

where R_(X) (●) is the rate distortion function for the original sourceX without side information.

In order to translate the performance of a feedback system to the systemof the present disclosure without feedback, the receiver must producereliability information about the incremental redundancy that iscomparable or better than would have been achieved by simplytransmitting the incremental redundancy bits over the noisy channel (aswould have been performed in the feedback case); the latter would havebeen done following a request for these incremental redundancy bits by aNACK in the feedback system. Let the distortion for the incrementalredundancy bits for unsuccessful decoders in stage j−1 transmitteddirectly over the channel be D_(U).

In the standard problem formulation in the articles of Verdu andWeissman, Perron and Diggavi, as well as that of A. D. Wyner and J. Ziv.“The Rate-Distortion Function for Source Coding with Side Information atthe Decoder” IEEE Trans. Inf. Theory, 22(1):1-10, January 1976., theside information values of X for the successful decoders are included inthe distortion computation. The distortion of the X values for theredundancy bits meant for the successful decoders is D_(S)=0 for anyuseful distortion measure since they are provided perfectly as sideinformation. Thus, in the example where the erasure probability is p weneed the overall distortion to be

D=pD _(U)+(1−p)D _(S) =pD _(U)   (12)

to ensure that the incremental bits for the unsuccessful decoders havethe desired distortion of D_(U). Looking again at R_(X|Y) (D) of Eq.(11), we find that the result simplifies to pR_(X) (D_(U)) so that, notsurprisingly, the overall rate required is simply the original ratedistortion function (without side information) multiplied by thefraction of bits that were not already provided as side information.Thus, information theory confirms that it is possible to provide asource coding solution that allows recovery of the incremental bits ofthe unsuccessful decoders with the same distortion and the same rate aswould have been possible if we knew at the transmitter which bits wererequired by the unsuccessful decoders.

To see this, consider applying this source coding theorem to a largenumber L of parallel decoders without feedback and comparing the amountof redundancy transmitted to those same L decoders operating withfeedback. Suppose that we are at a stage where the probability that eachdecoder requires incremental redundancy is p. For the system withfeedback, the expected total amount of redundancy over the L decoders isLp

(I_(j))R_(X)(D_(U)), and as we have L decoders times the probability pthat they require feedback times

(I_(j))R_(X)(D_(U)) bits per decoder that sends a NACK. For the Lparallel decoders without feedback, Theorem 18 from the previously citedS. Verdu and T. Weissman (“The Information Lost in Erasures”) alsoyields L

(I_(j)) pR_(X)(D_(U)) for the resulting source coding with sideinformation problem.

One appropriate distortion metric for the likelihoods of the incrementalredundancy is log loss distortion. Since the rate distortion functionfor log loss distortion is H(X)-D, and the expected distortion obtainedfrom transmitting X directly on the channel is H(X|Y) we arrive at theresult that the best possible compression of the incremental redundancyto achieve the same log loss distortion as would have been obtained bytransmitting the bits directly on the channel is exactly the channelcapacity H(X)-H(X|Y).

Thus, as with the well-known case of compressing a Gaussian source undersquared-error distortion and transmitting it over the Gaussian channel,there is no benefit asymptotically in the case with feedback to separatecompression and channel coding of incremental redundancy over simplytransmitting the redundancy directly on the channel. However, sourcecoding is one valid approach for compressing the incremental redundancyfor the system without feedback, in view of the side information thatwill be available at the receiver. Log-loss distortion is one possibledistortion metric for this source coding approach. One aspect of usingnumerous parallel decoders is that there are enough incremental bits tomake source coding a practical possibility, as has been noted by variousresearchers.

2.8 Joint Source-Channel Approaches

While separation of source and channel coding is an optimal approachfrom the perspective of information theory, embodiments of the presentdisclosure can also use multiple joint source-channel approaches thathave practical appeal to solving the problem of providing thereliabilities of incremental redundancy bits for the unsuccessfuldecoders at each stage in an efficient manner.

Tornado Codes, LT Codes, and Raptor Codes all address coding forerasures, but focus on the noiseless erasure channel. We note inparticular the discussion of Tornado Codes (M. G. Luby, M. Mitzenmacher,M. A. Shokrallahi, and D. A. Spielman. “Efficient erasure correctingcodes.” IEEE Trans. on Info. Th., 47(2): 569-584, Feb. 2001) for whichthe case of erasures only to systematic bits (on an otherwise noiselesschannel) was considered and referred to as the partial-erasure channel(A. Shokrollahi in “Raptor codes.” IEEE Trans. on Info. Th.,52(6):2551-2567, June 2006.). Under the partial erasure channel model,systematic bits and linear parity bits (typically in multiple stages)are both transmitted, but only systematic bits are erased. In theseworks, only the noiseless erasure channel is considered whereas thesituation of the present disclosure includes an AWGN channel.

Protograph-based codes and in particular protograph-based Raptor-like(PBRL) LDPC codes for the AWGN channel can be designed for the two-modechannel that is created by the proposed system. As shown in FIG. 6, inthis two-mode channel, the incremental bits themselves (or “precoded”versions of these bits) are effectively transmitted through an erasurechannel (where erasures correspond to unsuccessful decoders), while thelinear combinations of these precoded bits are transmitted over the AWGNchannel. As in the article from T.-Y. Chen, K. Vakilinia, D. Divsalar,and R. D. Wesel, “Protograph-based raptor-like LDPC codes”, IEEEtransactions on communications. IEEE Trans. Commun., 63(5):1522-1532,May 2015, the reciprocal channel approximation (RCA) algorithm asdiscussed by S. Y. Chung in “On the construction of somecapacity-approaching coding schemes.” PhD thesis, MIT, Cambridge, Mass.,2000, and by D. Divsalar, S. Donlinar, C. R. Jones, and Kenneth Andrewsin “Capacity-approaching protograph codes.” IEEE J. Sel. Areas Commun.,27, No. 6:876-888, August 2009 can be used to guide the design of theprotograph. RCA provides a fast and accurate approximation to thedensity evolution algorithm originally proposed by Richardson et al. inT. J. Richardson and R. L. Urbanke. “The capacity of low-densityparity-check codes under message passing decoding.” IEEE Trans. Inf.Theory, 47, No.2:599-618, February 2001, and in T. J. Richardson, M. A.Shokrollahi, and R. L. Urbanke. “Design of capacity-approachingirregular low-density parity-check codes.” IEEE Trans. Inf. Theory, 47,No.2:618-637, February 2001.

Experimental results show that the deviation of RCA from the exactdensity evolution threshold is less than 0.01 dB. Like in the T. Y. Chenarticle “Protograph-based raptor-like LDPC codes” we used RCA to designprotographs for rate-compatible code families within 0.4 dB of thenormal approximation on finite-length performance for a blocklength of16,384 at FER 10⁻⁵.

Embodiments of the feedback-less technology utilize protographs designedusing a hybrid version of RCA to characterize performance on thetwo-mode channel. Because the goal of this code is to deliver thereliabilities of these bits to the parallel decoders, the protographsshould be designed to optimize the messages (maximize the likelihoods)or minimize the log-loss distortion at a fixed SNR rather than to decodecorrectly at the lowest possible SNR. The hybrid RCA should use bothparameters of the erasure probability and SNR to optimize theconnections in the PBRL-structured protograph.

The recent results of Mitchell et al. in D. G. M. Mitchell, M.Lentmaier, A. E. Pusane, and D. J. Costello. “Randomly puncturedspatially coupled LDPC codes.” In Int. Symp. on Turbo Codes andIterative Information Processing (ISTC), pages 1-6, Bremen, August 2014,and D. G. M. Mitchell, M. Lentmaier, A. E. Pusane, and D. J. Costello.“Approximating decoding thresholds of punctured LDPC code ensembles onthe AWGN channel.” In IEEE Int. Symp. on Info. Theory, pages 421-425,Hong Kong, June 2015, show how the density evolution threshold of amother code can be used to characterize the performance of that code forthe full range of random erasure probabilities provides a useful tool inthis analysis.

2.9 A Peeling Decoder Approach

The above discussion treats the incremental-redundancy decoder as aone-shot device. However, a more flexible approach which can be employedby at least some embodiments of the present disclosure is to allow theincremental-redundancy decoder the ability to refine its results basedon successful decoding during the current stage. With the additionalside information from successful decoders in the current stage, theincremental-redundancy decoder can recover additional redundancy formore parallel decoders. This is directly analogous to the “peelingdecoder” described in M. G. Luby, M. Mitzenmacher, M. A. Shokrallahi,and D. A. Spielman. “Efficient erasure correcting codes.” IEEE Trans. onInfo. Th., 47(2):569-584, February 2001., which is at the heart ofiterative erasure decoding. In the above article, each success of thepeeling decoder unlocks an additional noiseless bit or frame. In thepresent disclosure, each time this peeling decoder is successful, newincremental redundancy for a parallel decoder is seen as if it were sentdirectly over the AWGN channel.

The recent work of Zeineddine and Mansour in “Inter-frame coding forbroadcast communication.” IEEE J. Select. Areas Commun.—Recent Advancesin Capacity-Approaching Codes, To Appear 2016, demonstrates thefeasibility of this peeling-decoder approach for recovering incrementalredundancy. This approach is one technique for providing incrementalredundancy only to the decoders that need it. Zeineddine and Mansourconsider all segments of incremental redundancy to be of the same lengthand to be interchangeable from the perspective of the decoder.Considering Table 1, it can be seen that the optimal increments for themiddle 6 or 7 stages are all around eleven bits. Thus embodiments of ourfeedback-less technology may combine some or all stages of incrementalredundancy as described in the Zeineddine and Mansour article.

2.10 Variable-length Codes that approach Capacity

At the crux of the present disclosure is the combination ofvariable-length codes that approach capacity with short averageblocklengths by using incremental redundancy (that would previously havebeen controlled by feedback) and a technique for delivering thatincremental redundancy without using feedback. Thus, the presentdisclosure requires the existence of variable-length codes that approachcapacity with short average blocklengths by using an initialtransmission followed by up to m−1 transmissions of incrementalredundancy called increments.

The NB-LDPC code with m=10, whose performance was shown in FIG. 1,provides an existence proof that a variable-length code exists that cancome close enough to capacity at a short-enough blocklength to enablethe present disclosure. This code achieves 93% of the capacity of theBI-AWGN channel for k=280, where 280 information bits are used alongwith an 8-bit CRC to determine that decoding was successful. For thissystem, the target variable-length codeword error rate is 10⁻³ and theaverage blocklength is below 500 symbols.

The 1024-state TBCC code with ROVA explored in “A. R. Williamson, T.-Y.Chen, and R. D. Wesel. Variable-length convolutional coding for shortblocklengths with decision feedback. IEEE Trans. Commun.,63(7):2389-2403, July 2015.” provide a second example, achieving asmaller percentage of capacity but at an even shorter averageblocklengths. With k=64 this code achieves 82% of capacity with m=5, 84%of capacity with m=16 , and 86% of capacity with m=32 .

The PBRL codes in “T. Y. Chen, K. Vakilinia, D. Divsalar, and R. D.Wesel, Protograph-based Raptor-Like LDPC codes. IEEE Trans. Commun.,63(5):1522-1532, May 2015.” closely approach capacity for theirblocklengths, and provide another possible family of variable-lengthcodes that can be used in embodiments of the present disclosure.

In any case, to practice the present disclosure, a variable-length codemust be identified that approaches capacity (i.e., achieves at least 75%of the Shannon capacity but preferably more than 90% of capacity) with ashort average blocklength (average blocklength of less than 1500 symbolsbut preferably less than 500 symbols) by using incremental redundancythat is required only when a previous decoding stage has failed. Thisvariable length code is utilized to send multiple k-bit messages inparallel and encodes the incremental redundancy of these messagestogether in a way that the receiver can, without using feedback, accessthe needed incremental redundancy for all or almost all of thosedecoders that fail to decode in a previous stage, while still achievinga throughput close to that of the system that uses the variable-lengthcode with feedback and thus still closely approaches capacity.

2.10 Additional Embodiments

As additional embodiments, we consider two example designs that utilizethe interframe coding approach of Zeineddine and Mansour and a TBCC thatachieves over 80% of the Shannon Capacity with an average blocklengththat is less than 150 symbols.

The performance of a VL code with incremental redundancy (IR) andfeedback is characterized by the throughput rate R_(t) ^((FB)) achievedwhile not exceeding a target (codeword) undetected error rate P_(UE),where

${R_{t}^{({FB})} = \frac{E\lbrack K\rbrack}{E\lbrack N\rbrack}},$

where E[K] represents the expected number of information bitstransmitted by the VL code, and E[N] represents the average number oftransmitted symbols (i.e., average blocklength) of the VL code. Indescribing these embodiments, K is a random variable describing how manybits were successfully transmitted by a variable-length code so that Ktakes on values of k or zero. The unit of throughput (or rate) indiscussing these embodiments is bits per transmitted symbol. Theprobability of failure P_(j) is the probability that a VL codewordcannot be decoded even at its maximum length when all IR has beenreceived. Ideally, R_(t) ^((FB)) should be as close to the channelcapacity as possible.

Let

_(j) represent the length (i.e., number of symbols) of the j^(th)transmission X₁ for a VL codeword, where X represents a multi-symbolsequence. Also let m represent the maximum number of transmissionsallowed. After the initial transmission X₁ of length

, the transmitter sends IR transmissions X_(j) of length

_(j), j∈{2,

m} if requested by the receiver.

Let p.m.f. δ={δ(1), . . . δ(ω), . . . δ(m+1)} represent the probabilitythat a codeword requires ω transmissions including the initialtransmission to decode successfully, i.e., ω total transmissions. Thusfor 1≤ω≤m, δ(ω) is the probability that a codeword was successfullydecoded after ω transmissions, and δ(m) is the probability of failure todecode even after the initial transmission and all m−1 available IRtransmissions are received. Let

_(FB) be the target failure probability P_(f) of a feedback system. Therequirement δ(m+1)≤ϵ_(FB) must be satisfied.

The p.m.f. δ(ω) plays a critical role in the design of the system. In atleast one embodiment, we obtain δ(ω) for our target shortvariable-length code through Monte-Carlo simulation. It should be notedthat our design procedure places no restriction on δ(ω).

With this notation, E[K]=k×(1−δ(m+1)−P_(UE)) where k represents thenumber of information bits in the message W encoded in a VL codeword.The expected blocklength E [K] of the VL code is

${E\lbrack K\rbrack} = {{\sum\limits_{j = 1}^{m}\left( {{\delta (j)} \times {\sum\limits_{i = 1}^{j}l_{i}}} \right)} + {{\delta \left( {m + 1} \right)} \times {\sum\limits_{i = 1}^{m}{l_{i}.}}}}$

For these embodiments we focus on the system described in “A. R.Williamson, T.-Y. Chen, and R. D. Wesel. Variable-length convolutionalcoding for short blocklengths with decision feedback. IEEE Trans.Commun., 63(7):2389-2403, July 2015.” in which a 1024-state TBCC usingfeedback from a reliability output Viterbi algorithm (ROVA) achieve 82%of 2 dB BI-AWGN capacity with m=5 transmissions with an averageblocklength of less than 150 symbols and a target codeword error rate of10⁻³. In this disclosure, all simulations demonstrating this embodimentare conducted using the 2 dB BI-AWGN channel, and by way of example, allsystems in these embodiments have a target failure rate

_(FB)=10⁻³.

In these embodiments, the IR transmissions may be used in any order(guaranteed by the pseudorandom puncturing of Williamson et. al.,) andhave constant length, with for example

₁=

₂=. . . =

_(m)=

_(Δ). The optimal lengths

_(j) for IR transmissions in a variable-length code with m possibletransmissions as determined using SDO, in general, are variable.However, our simulations indicate that a capacity can still beapproached with the suboptimal solution that requires constantincrements (CI). Table 2 shows a comparison between the bestvariable-increment (VI) and CI system designs for the 1024-state TBCCwith k=64 information bits and m=5.

Table 2 shows that the best CI design achieves 99% of the R_(t) ^((FB))of the VI designs, so that the CI requirement does not significantlyaffect performance. To accommodate some additional failure mechanisms inthe system without feedback, the increment size is increased by one to16 for the CI system used in our simulations in the next section,identified as “Actual” in Table 2. This CI design still achieves 98.5%of the R_(t) ^((FB)) of the VI designs. The increment sizes of 15 or 16is considered here to approach capacity with a short blocklength, andare significantly smaller than those contemplated in Zeineddine andMansour, which has increment sizes on the order of 550 symbols, asutilized to combat fading.

2.10.1 Transmitter

The transmitter sends uses L VL codewords to send in parallel L messagesW₁,

, W_(L), each containing k bits of information. The VL code we use forat least one embodiment of our design is the k=64, 1024-state TBCC withm=5 from Williamson et. al. Each message W_(i) is encoded by the VLencoder to produce the initial transmission X₁ ^((i)) with length

₁ and the IR sequences X_(j) ^((i)) for j=2,

m all of length

_(Δ). The initial transmissions X₁ ^((i)) are transmitted directly overthe channel, but the IR sequences X_(j) ^((i)) for j=2,

m are not directly transmitted over the channel.

Instead, a second code (the inter-frame code of Zeineddine and Mansour)combines increments from different VL codes through exclusive-oroperations. For example, the first transmitted linear combination mightbe I₁=X₁ ⁽¹⁾⊕X₂ ⁽²⁾⊕X₁ ^((n)), where ⊕ indicates bitwise exclusive-or.

FIG. 7 illustrates a diagram of the overall process including the VLencoders and the inter-frame code. A total of n_(i) such linearcombinations are created and transmitted over the channel. Each linearcombination has the length l_(Δ), since it is a linear combination of IRsequences all of length l₆₆.

It is instructive to compare the rate of this feedforward (i.e., nofeedback) system to the original feedback system using the same CI VLcodes. The rate of the feedback system is R_(t) ^((FB)) as defined inEq. (1). With the feedforward system, L·l₀+n_(i)·l_(Δ) symbols aretransmitted for the n_(c) k-bit messages encoded by the system, so therate is

$R_{t}^{({FF})} = {\frac{{Lk}\left( {1 - ɛ_{FF} - P_{UE}} \right.}{{Ll}_{0} + {n_{i}l_{\Delta}}}.}$

Our inter-frame code design uses the p.m.f. δ(ω) to find a degreedistribution for an LDGM code that can provide the needed incrementalredundancy to the L decoders of the parallel VL codes with the smallestvalue of n_(i), while still achieving a failure rate

_(FF) that is below the target.

The inter-frame code has the Tanner graph that is a generalization ofthe low-density generator matrix (LDGM) codes described, for example, in“J. F. Cheng and R. L. McEliece, Some high-rate near capacity codes forthe Gaussian Channel,” in Proc. of 34^(th) Allerton Conference onCommunication, Control, and Computing.” Let each of the n_(o) VLcodewords be represented by a systematic node of Cheng and McEliece, andeach of the n_(i) linear combinations of IR be represented by a paritynode of Cheng and McEliece. The rate of the LDGM code is defined as

$R_{i} = {\frac{L}{L + n_{i}}.}$

From Eq. (3) and Eq. (4), R_(t) ^((FF)) is a function of the R_(i) asfollows

$R_{t}^{({FF})} = {\frac{k\left( {1 - ɛ_{FF} - P_{UE}} \right)}{l_{0} + {\left( {R_{i}^{- 1} - 1} \right)l_{\Delta}}}.}$

Thus increasing R_(i) also increases R_(t) ^((FF)).

2.10.2 Receiver

The following describes the generalized peeling decoder utilized in atleast one embodiment of the present disclosure for decoding theincremental redundancy code. At the beginning of each iteration, everyundecoded VL code attempts to decode with its (noisy) initialtransmission X₀ and any (noisy) IR symbols that are available to it fromthe incremental redundancy decoder. The successfully decoded VL codescalculate their remaining IR sequences X_(j) and send these to theparity nodes that are connected to the systematic nodes corresponding tothose VL codes.

When a parity node has received IR sequences X_(j) from all connectedsystematic nodes except one remaining systematic node r, the linearcombination of all received X_(j) sequences is used to apply the correctsign to the reliability of each bit of the combination-IR sequenceI_(j), and the resulting noisy IR sequence X_(j) ^((r)), is provided tothe VL decoder for code r with reliabilities equivalent to what wouldhave been received if X_(j) ^((r)) had been directly transmitted overthe channel. This process repeats until no new VL codes can be decoded.

Zeineddine and Mansour analytically derived inter-frame code degreedistributions for the specific case of a geometric δ(ω) based on theirapplication of combating fading on a broadcast channel.

In this example of the present disclosure, we focus on a single-userpoint-to-point channel with a well-defined channel capacity. For thisapplication δ(ω) will not be a geometric distribution. A differentialevolution process is utilized to construct the generalized LDGM degreedistribution for any δ(ω), and designing regular and irregular LDGMsaccording to these degree distributions, and simulate their performanceusing a generalized peeling decoder.

2.10 Design of the Incremental Redundancy Code

In this section, a design methodology is provided to obtain the LDGMcodes that operate as incremental redundancy codes in an embodiment ofthe overall system. Given m and a channel distribution δ(ω), it is shownhow to obtain LDGM codes that enable the overall system to approach thethroughput of the original feedback codes (and thus the channelcapacity), but without feedback.

In Zeineddine and Mansour, a message passing algorithm is considered inthe analysis of the inter-frame code. We analyze a peeling decoder, “M.G. Luby, M N. Mitzenmacher, M. A. Shokrollahi, and D. A. Spielman,“Efficient Error correcting codes” IEEE Trans. Inf. Theory, vol. 47, no.2, pp 569-584, February 2001.” to reduce the decoding complexity of theactual system.

A density-evolution approach can be used to characterize the asymptoticdecoding failure rate and the number of iterations required to achieveit. For this analysis, the messages are modeled from parity nodes tosystematic nodes as a single bit: a 1 if an IR sequence can be providedto a VL decoder and a 0 otherwise. Similarly, a message of 1 from asystematic node to a parity node represents the event where thesystematic node can send an IR sequence X_(j) to the parity node and a 0otherwise.

Let z be the probability that a randomly picked parity-to-systematicmessage is 0 in the current iteration. Let g(z) describe the probabilitythat a randomly picked systematic-to-parity message is 0 given z in thecurrent iteration, and f(z) describe the probability that a randomlypicked parity-to-systematic message is 1 in the next iteration. For anydecoder, there always exists a z* such that

1−f(z)<z, ∀∈[z*,1]  (13)

For any pair of systematic-node edge degree distribution polynomialλ(z)=Σ_(i)λ_(l)x^(i−1) and parity-node edge degree distributionpolynomial p(x)=Σ_(i)p_(i)x^(i−1), f(z) can be calculated as

$\begin{matrix}{{{f(z)} = {{\sum\limits_{i}{\rho_{i}\left( {1 - {g(z)}} \right)}^{i - 1}} = {\rho \left( {1 - {g(z)}} \right)}}},} & (14)\end{matrix}$

and g(z) can be calculated as

$\begin{matrix}{{g(z)} = {{\sum\limits_{d}{\lambda_{d} \cdot {g_{d}(z)}}} = {\sum\limits_{d}{\lambda_{d} \cdot {\left( {\sum\limits_{\omega}{{\delta (\omega)} \cdot {g_{d;\omega}(z)}}} \right).}}}}} & (15)\end{matrix}$

Because we analyze a generalized peeling decoder rather than amessage-passing decoder, our calculation of g_(d:ω)(z) below isdifferent from Zeineddine and Mansour:

$\begin{matrix}{{g_{d;\omega}(z)} = {\sum\limits_{j = 0}^{\min {({{\omega - 1},d})}}{\begin{pmatrix}d \\j\end{pmatrix} \cdot \left( {1 - z} \right)^{j} \cdot {z^{d - j}.}}}} & (16)\end{matrix}$

Similarly, the systematic node error rate can be calculated as

$\begin{matrix}{{{G(z)} = {\sum\limits_{d}{\Lambda_{d} \cdot {g_{d}(z)}}}},} & (17)\end{matrix}$

where Λ(x)=Σ_(i)Λ_(i)x^(i−1) is the systematic-node degree distributionpolynomial.

It should be appreciated that in an actual decoder of the inter-framecode, the degree distributions and the channel distribution δ(ω) changeafter every iteration. As a result, Eq. (13)-(16) provide a precisecharacterization of the decoding process only for the first iteration.For later iterations, the analysis in this section is an approximation,and performance of the differential evolution below is improved byreplacing the approximation with the actual density evolution resultsfor the generalized peeling decoder we have developed.

By way of example and not limitation, we use the approach ofdifferential evolution (DE) “A. Shokrollahi and R. M. Storn, “Design ofefficient erasure codes with differential evolution” in “DifferentialEvolution: a practical approach to global optimization. Springer VerlagBerlin Heidelberg, 2005, ch. 7 pp 413-426” to optimize the degreedistributions for the generalized LDGM code. The asymptotic error rate(of systematic nodes) is minimized. The constraints of the optimizationare R_(i), the maximum systematic degree L, the maximum parity degree R,and the target error rate. There are L+R variables, L+R−3 of which needto be optimized. The parameter vector is defined to be p={λ₂, . . . ,λ_(L−1), p₂, . . . p_(R)}. The process has the following general steps:

(1) Initialization: First, a pair (λ_(C)(x), p_(C)(x)) that satisfiesthe degree and rate constraints is generated, and for example, theChebyshev center of the L+R−3 dimension feasibility space is a workablechoice. Second, an initial population of size N_(p) is generatediteratively using the hit-and-run sampler “R. L. Smith, “Efficient MonteCarlo procedures for generating points uniformly distributed overbounded regions,” Operations Research, vol. 32, no. 6, pp1296-1308,November 1984.” as follows. From point p_(i), the sampler chooses arandom direction. A feasible random point in that direction is selectedas the new point p_(i+1). In the following we use G to represent thegeneration number of a population, and the initialization population hasG=0.

For each point in the population, z* is calculated using Eq. (13)-(16),and the asymptotic error rate of systematic nodes G(z*) is determinedusing Eq. (17).

From the definition of z and f(z), we have the following relationshipbetween the probability z in consecutive iterations:

z _(i+1)=1−f(z _(i)),

where the subscript i represents the i th iteration. The number ofiterations required to achieve z* can be obtained by setting z₀=1, andfinding the smallest i that satisfies z_(i)=z*.

(2) Mutation: For each point p_(i) in the generation G(i=0,1, . . . ,N_(p)−1), a generation G+1 point v_(i,G+1) ={v_(0i,G+1), v_(1i,G+1), . .. , v_((L+R−3)i,G+1)} is produced using

v _(i,G+1) =p _(best,G)−0.5·(p _(r1,G) −p _(r2,G) +p _(r3,G) −p_(r4,G)).

The point p_(best,G) represents the point in the G th generation thathas the lowest G(z*), and r_(l), r₂, r₃ and r₄ are randomly chosen over[0,N_(p)−1].

(3) Recombination: A new generation G+1 point u_(i,G+1)={u_(0i,G+1),u_(1i,G+1), . . . , u_((L+R−3)i,G+1)} is generated from v_(i,G+1) usingthe following procedure for j=0,1, . . . ,L+R−4:

$\begin{matrix}{u_{{ji},{G + 1}} = \left\{ {\begin{matrix}v_{{ji},{G + 1}} & {{{{{if}\mspace{14mu} U_{{rand},j}} \leq {P_{CR}\mspace{14mu} {or}\mspace{14mu} j}} = {K(i)}},} \\p_{{ji},G} & {otherwise}\end{matrix}.} \right.} & \;\end{matrix}$

Here U_(rand,j) ∈[0,1] is a randomly generated value following a uniformdistribution for each j, K(i) is a randomly chosen index over[0,N_(p)−1] that stays the same for point i, and P_(CR) is the crossoverprobability.

(4) Selection: In this step, u_(i,G+1) or p_(i,G) is selected asp_(i,G+1). For designing the inter-frame code, the selection criteriaare G(z*) and the number of iterations to achieve G(z*). Between thetwo, the error rate is favored as it indicates the code's correctioncapability. The point with the lower G(z*) and smaller, or the samenumber, of iterations is chosen.

After selecting the (G+1) th generation, the process starts frommutation again. The iterative process stops after a certain number ofgenerations. If the error rate after the maximum number of iterations ishigher than the target error rate ϵ, then the preset rate R_(i) is toohigh.

In this embodiment, the maximum systematic node degree is L=4 (sincem=5). The maximum parity node degree is R=10 for our irregular code. Inthis example, we set the population size N_(p)=500, the maximum numberof generations as 50, and the crossover probability P_(CR)=1. It shouldbe appreciated with this example, and others throughout the text, thatother sets of values may be utilized for specific applications withoutdeparting from the teachings of the present disclosure.

If the incremental redundancy code is designed using “Best” CI lengthsin Table 2, the error rate would very likely be higher than the target.Therefore, we use the “Actual” CI lengths in Table 2. The target errorrate of the differential evolution process is set accordingly toϵ=6×10⁻⁴, slightly above the failure rate of the VL code using the CI“Actual” lengths. We use the Progressive edge growth (PEG) algorithm ofHu et al., “X.-Y. Hu, E Eleftheriou, and D. M. Arnold, “Regular andirregular progressive edge growth Tanner graphs,” IEEE Transactions onInformation Theory, vol. 51, no. 1, pp 386-398, January 2005” toconstruct the incremental redundancy codes based on the degreedistribution that we obtain. In our degree distributions obtained fromDE, we observed that the systematic-node degree distributions haveextremely high percentage (>99.99%) of degree L=4 nodes. We constructthe LDGM graph from the perspective of parity nodes, then modified thegraph from the perspective of systematic nodes, while maintaining thegirth, to make sure all systematic nodes have a degree of 4. The highestrate regular and irregular incremental redundancy codes designedaccording to the procedure described above are now presented. Both codeshave 100,000 systematic nodes. Simulations of the incremental redundancycoder are conducted using the δ(ω) generated from simulating a k=641024-state TBCC under 2 dB BI-AWGN channel with m=5 transmissions. Theresulting δ(ω) was found to be the following:

δ={0.333036, 0.448600, 0.182245, 3.15894×10⁻², 4.02445×10⁻³,5.05161×10⁻⁴}.

The regular code ideally has only degree-4 systematic nodes and degree-3parity nodes, and rate R_(i)=3/7 ≈0.428571. The actual code matrix has asystematic-node degree distribution λ={0, 0, 0, 1} and a parity-nodedegree distribution p={0, 0, 0.0001, 0.9999}, for example, it is veryslightly irregular. The rate of the code described by our actual codematrix is R_(i)=0.428578. The rate of overall feedforward system R_(t)^((FF))=0.494503. The simulated failure probability of the inter-framecode is 6.691×10⁻⁴.

The highest-rate irregular code we designed has a theoretical rateR_(i)=0.48. From the differential evolution process, the systematic-nodedegree distribution is λ={0,0,0,1,},and the parity-node degreedistribution is

p={2.73614×10⁻³, 2.36956×10⁻³, 0.445366, 0.213356, 0.188564,2.68009×10⁻², 4.89986×10⁻²,1.21422×10⁻³, 1.77664×10⁻², 3.15029×10⁻²}.

The actual code matrix has λ={0,0,0,1} as its systematic-node degreedistribution. The parity-node degree distribution is

p={2.6575×10⁻³, 2.4735×10⁻², 0.4420125, 0.21653, 0.1872125, 2.8155×10⁻²,

4.84225×10⁻², 1.46×10⁻³, 1.9215×10⁻², 2.96×10⁻²}. The rate of the codedescribed by the code matrix is R_(i)=0.480008. The rate of thefeedforward system R_(i) ^((FF))=0.510182. The failure rate of theinter-frame code is 8.979×10⁻⁴.

Table 3 compares the results with the best feedback system under thesame constraints. The irregular feedforward system achieves more than95% of the best R_(i) ^((FB)) for m=5. In addition to the densityevolution approximation analysis in a later section, we also determinedthe exact density evolution for both the regular and irregular code.

FIG. 8 compares the density evolution prediction of the codeword errorrate at each iteration with the simulation result. The regular code'sresult is very closely approximated by the density evolution. Thesimulation result of the irregular code is also close to densityevolution, and the prediction of the asymptotic codeword error ratematches with the simulation result in both cases.

While the embodiments presented here used 100,000 systematic nodes andthus n_(c)=100,000, we have used the same approach to design systemswith n_(c)=10,000 and n_(c)=1,000.

3. Summary of Embodiments

This disclosure describes a new system that combines variable-lengthcodes that approach capacity with short average blocklengths by usingincremental redundancy (that would previously have been controlled byfeedback) and a technique for delivering that incremental redundancywithout using feedback. Thus, the present disclosure requires theexistence of variable-length codes that approach capacity with shortaverage blocklengths by using incremental redundancy.

The new system approaches capacity by harvesting the ergodicity benefitsof long blocklengths while achieving the decoder complexity of ashort-blocklength code. The transmitter first sends the initialtransmission for multiple short blocklength codes without additionalprocessing. Some of these short-blocklength codes can be redundant inthat they are produced by a fixed-length packet-level erasure code. Thesubsequent stages that follow the initial transmission are produced byan incremental redundancy encoder that combines and compresses theincremental redundancy associated with a stage (or in some embodimentswith multiple stages). The incremental redundancy encoders associatedwith different stages will typically be different and have differentrates because with each stage more of the short blocklength codes aresuccessfully decoded. However, the number of symbols or bits transmittedfor each stage or group of stages is a fixed number that is establishedbased on the amount of redundancy needed for that stage which in turn isbased on the minimum number of decoders successful in previous stages,which is a function of channel quality. Also, the lengths of theincremental redundancy associated with different stages of theshort-blocklength encoder are typically different, although they mightbe the same in some embodiments.

Similar to network coding, the incremental redundancy encoder may employlinear combinations of the incremental redundancy stages of the shortblocklength codes. Other embodiments of the incremental redundancyencoder include separately compressing the incremental redundancy usinglossy compression under a distortion criteria, such as log lossdistortion. This compressed incremental redundancy would then betransmitted with additional channel coding if needed. In still otherembodiments, the incremental redundancy encoder can employ jointsource-channel encoding using, for example, a protograph-based LDPC codedesigned for a channel that has both noisy symbols that correspond tothe symbols that are actually transmitted over the channel noiselesssymbols that may be erased. The noiseless symbols correspond toincremental redundancy for the short-blocklength codes that havesuccessfully decoded and the erasures correspond to incrementalredundancy for the short-blocklength codes that have not yetsuccessfully decoded.

At the decoder, after the initial transmission (the first stage ofincremental redundancy) is received, decoding is attempted for eachshort-blocklength code. In the second and subsequent stages ofincremental redundancy, the incremental redundancy decoder providesshort blocklength decoders that have not yet successfully decoded withincremental redundancy to continue decoding. Once decoding is successfulfor a short-blocklength code, it provides information to the incrementalredundancy decoder. The incremental redundancy may not provide all ofthe short blocklength decoders with incremental redundancy at the sametime because in some embodiments it may use successful decodingoccurring during that stage to unlock incremental redundancy for some ofthe short blocklength codes.

The enhancements described in the presented technology can be readilyimplemented within various error correcting communication encoders anddecoders. It should also be appreciated that such encoders and decodersare preferably implemented to include one or more computer processordevices (e.g., CPU, microprocessor, microcontroller, computer enabledASIC, etc.) and associated memory storing instructions (e.g., RAM, DRAM,NVRAM, FLASH, computer readable media, etc.) whereby programming(instructions) stored in the memory are executed on the processor toperform the steps of the various process methods described herein.

Computer and memory devices were not depicted in the diagrams for thesake of simplicity of illustration, as one of ordinary skill in the artrecognizes the use of computer devices for carrying out steps involvedwith communications encoding and decoding. Computers may be utilized forcontrolling the encoders and/or decoders, as well as for performingelements of the encoding and decoding, without limitation. The use ofcomputers does not preclude the use of other forms of circuitry, asthese are often combined in various embodiments of encoders/decoders.The presented technology is non-limiting with regard to memory andcomputer-readable media, insofar as these are non-transitory, and thusnot constituting a transitory electronic signal.

Embodiments of the present technology may be described herein withreference to flowchart illustrations of methods and systems according toembodiments of the technology, and/or procedures, algorithms, steps,operations, formulae, or other computational depictions, which may alsobe implemented as computer program products. In this regard, each blockor step of a flowchart, and combinations of blocks (and/or steps) in aflowchart, as well as any procedure, algorithm, step, operation,formula, or computational depiction can be implemented by various means,such as hardware, firmware, and/or software including one or morecomputer program instructions embodied in computer-readable programcode. As will be appreciated, any such computer program instructions maybe executed by one or more computer processors, including withoutlimitation a general purpose computer or special purpose computer, orother programmable processing apparatus to produce a machine, such thatthe computer program instructions which execute on the computerprocessor(s) or other programmable processing apparatus create means forimplementing the function(s) specified.

Accordingly, blocks of the flowcharts, and procedures, algorithms,steps, operations, formulae, or computational depictions describedherein support combinations of means for performing the specifiedfunction(s), combinations of steps for performing the specifiedfunction(s), and computer program instructions, such as embodied incomputer-readable program code logic means, for performing the specifiedfunction(s). It will also be understood that each block of the flowchartillustrations, as well as any procedures, algorithms, steps, operations,formulae, or computational depictions and combinations thereof describedherein, can be implemented by special purpose hardware-based computersystems which perform the specified function(s) or step(s), orcombinations of special purpose hardware and computer-readable programcode.

Furthermore, these computer program instructions, such as embodied incomputer-readable program code, may also be stored in one or morecomputer-readable memory or memory devices that can direct a computerprocessor or other programmable processing apparatus to function in aparticular manner, such that the instructions stored in thecomputer-readable memory or memory devices produce an article ofmanufacture including instruction means which implement the functionspecified in the block(s) of the flowchart(s). The computer programinstructions may also be executed by a computer processor or otherprogrammable processing apparatus to cause a series of operational stepsto be performed on the computer processor or other programmableprocessing apparatus to produce a computer-implemented process such thatthe instructions which execute on the computer processor or otherprogrammable processing apparatus provide steps for implementing thefunctions specified in the block(s) of the flowchart(s), procedure (s)algorithm(s), step(s), operation(s), formula(e), or computationaldepiction(s).

It will further be appreciated that the terms “programming” or “programexecutable” as used herein refer to one or more instructions that can beexecuted by one or more computer processors to perform one or morefunctions as described herein. The instructions can be embodied insoftware, in firmware, or in a combination of software and firmware. Theinstructions can be stored local to the device in non-transitory media,or can be stored remotely such as on a server, or all or a portion ofthe instructions can be stored locally and remotely. Instructions storedremotely can be downloaded (pushed) to the device by user initiation, orautomatically based on one or more factors.

It will further be appreciated that as used herein, that the termsprocessor, computer processor, central processing unit (CPU), andcomputer are used synonymously to denote a device capable of executingthe instructions and communicating with input/output interfaces and/orperipheral devices, and that the terms processor, computer processor,CPU, and computer are intended to encompass single or multiple devices,single core and multicore devices, and variations thereof.

From the description herein, it will be appreciated that the presentdisclosure encompasses multiple embodiments which include, but are notlimited to, the following:

1. A transmitter apparatus in a communications system, said transmittercomprising: (a) one or more variable-length encoders for avariable-length code configured to take as input a k-bit message and toproduce as output an initial transmission sequence of length

₁ symbols and one or more increments having lengths

₂,

,

_(m) symbols, such that if said initial transmission sequence and saidincrements were transmitted based on non-active ACK/NACK feedback, saidvariable-length code would approach capacity by exceeding at least aspecified percentage of Shannon capacity of a specified point-to-pointchannel, without using feedback to control the transmission of saidincrements; and (b) an incremental redundancy encoder configured tocompress and encode increments produced by said one or morevariable-length encoders to produce a common pool of incrementalredundancy to be made available for all variable-length codes at areceiver; (c) wherein said transmitter apparatus is configured toutilize said one or more variable-length encoders to produce saidinitial transmission sequences of a plurality L of said k-bit messages,which are transmitted to a receiver over a channel between saidtransmitter and said receiver; (d) wherein said transmitter apparatus isconfigured to transmit to said receiver over said channel between saidtransmitter and said receiver a number of transmitted symbols,corresponding to a common pool of redundancy for the plurality L of saidk-bit messages; (e) wherein said number of transmitted symbolscorresponding to the common pool of redundancy approximates a number ofsymbols of incremental redundancy that would have been sent by Lindependently operating variable-length encoders if said increments weretransmitted based on non-active ACK/NACK feedback on said specifiedpoint-to-point channel; (f) wherein said transmitter apparatus isconfigured to approach Shannon capacity of a specified point-to-pointchannel, having a rate similar to that achieved by capacity-approachingvariable-length code when utilized with non-active ACK/NACK feedback;and (g) wherein probability of either failing to decode a k-bit message,or decoding a k-bit message in error is below a target value.

2. The transmitter apparatus of any preceding embodiment, wherein saidinitial transmission and said one or more increments have lengths

₁,

,

_(m) that have been optimized using sequential differentialapproximation.

3. The transmitter apparatus of any preceding embodiment, wherein saidvariable-length encoder is configured for utilizing a low-densityparity-check code.

4. The transmitter apparatus of any preceding embodiment, wherein saidvariable-length encoder comprises a tail-biting convolutional encoderwith rate-compatible puncturing.

5. The transmitter apparatus of any preceding embodiment, in which theincrement lengths

₂,

,

_(m) are all less than 50.

6. The transmitter apparatus of any preceding embodiment, in which thevariable length code has an average blocklength of less than 500 symbolswere it to be used with non-active ACK/NACK feedback on the specifiedpoint-to-point channel.

7. The transmitter apparatus of any preceding embodiment, wherein saidvariable-length code achieves more than 90% of the Shannon capacity ofsaid specified point-to-point channel.

8. The transmitter apparatus of any preceding embodiment, wherein saidincremental redundancy encoder is configured to separately encode eachof the m−1 increments associated with a k-bit message, so that at stagej where j ∈{2,

m}, said incremental redundancy encoder is configured to take as input Lincrements, each of length

_(j) and to produce as output a portion of a common pool of incrementalredundancy associated with increment j.

9. The transmitter apparatus of any preceding embodiment, wherein saidincremental redundancy encoder is configured for jointly encoding atleast a portion of said m−1 increments associated with a k-bit message,so that in one of its operations said incremental redundancy encodertakes as input s groups of L increments associated with s consecutivestages, having lengths

_(j),

_(j+1),

_(j+s−1) to jointly produce a portion of a common pool of incrementalredundancy associated with increments j through j+s−1.

10. The transmitter apparatus of any preceding embodiment, wherein saidtransmitter apparatus is configured for being utilized in an opticalpoint-to-point channel.

11. The transmitter apparatus of any preceding embodiment, whereinnumber of symbols produced by said incremental redundancy encoder foreach stage depends on a probability that a variable length decoder willdetermine that it has successfully decoded before requiring incrementalredundancy associated with that stage as well as a length

_(j) of an increment for that stage and said number L of variable-lengthcodes with some overhead to account for uncertainty in said number ofvariable length decoders that determine successful decoding before thatstage.

12. The transmitter apparatus of any preceding embodiment furthercomprising: an additional encoder configured for protecting said k-bitmessages so that all user information is recovered even if some k-bitmessages are not successfully decoded by said variable-length decoders;and wherein said additional encoder is configured to take L×R_(e) k-bitmessages containing user information as input and produce as output saidplurality L of said k-bit messages encoded by said variable-lengthencoders so that probability that the full set of L×R_(e) k-bit messagesis not recovered successfully as a super-frame error rate, is below atarget value.

13. The transmitter apparatus of any preceding embodiment, wherein saidincremental redundancy encoding is accomplished in response to utilizingseparate source code and channel code, in which said source codecompresses said increments and said channel code protects compressedincrements.

14. The transmitter apparatus of any preceding embodiment, wherein saidincremental redundancy encoding is configured for utilizing a singleencoding operation that jointly compresses and sufficiently protectssaid increments to produce symbols ready for transmission.

15. The transmitter apparatus of any preceding embodiment, wherein saidtransmitter apparatus is configured to utilize said one or morevariable-length encoders to produce in parallel said initialtransmissions of a plurality L of said k-bit messages.

16. A receiver apparatus in a communications system, said receiverapparatus comprising: (a) one or more variable-length decoders for avariable-length code where variable length codewords are comprised of aninitial transmission sequence of length

₁ symbols by itself, or together with m−1 increments having lengths

₂,

_(m) symbols, such that if said initial transmission and said incrementswere transmitted based on non-active ACK/NACK feedback, saidvariable-length would approach capacity by exceeding at least aspecified percentage of Shannon capacity of a specified point-to-pointchannel, without using feedback to control the transmission of saidincrements; (b) wherein said variable-length decoders are configured forperforming multiple decoding attempts, in which each successive attemptutilizes as its input an input sequence of a previous attempt plus anadditional increment; (c) wherein said one or more variable-lengthdecoders is configured for producing for each attempt either a decodedk-bit message or an indication that no decoded k-bit message isavailable for that attempt; (d) an incremental redundancy decoderconfigured for receiving as input (d)(i) received symbols correspondingto a common pool of redundancy produced by an incremental redundancyencoder, and (d)(ii) increments produced by already-decoded k-bitmessages, and said incremental redundancy decoder is configured forproducing output increments for use by said variable-length decoders;(e) wherein said one or more variable-length decoders is configured tofirst attempt decoding of said plurality of L initial transmissions,each corresponding to a k-bit message, which are transmitted over achannel between the transmitter and the receiver, and continue untildecoding is successful; (f) wherein said one or more variable-lengthdecoders is configured for continuing to attempt decoding by avariable-length decoder whenever an additional increment is madeavailable from said incremental redundancy decoder for saidvariable-length decoder for that k-bit message; (g) wherein saidincremental redundancy decoder is configured for utilizing incrementscorresponding to each successfully-decoded k-bit message to subsequentlyproduce additional increments for said variable-length decoders until nomore new increments from successfully-decoded k-bit messages areavailable; (h) wherein said number of received symbols corresponding tothe common pool of redundancy approximates said number of symbols ofincremental redundancy that would have been sent by L independentlyoperating variable-length encoders if said increments were receivedbased on non-active ACK/NACK feedback from a receiver on that samechannel; (i) wherein a communications system utilizing said receiverapparatus approaches Shannon capacity of said specified point-to-pointchannel, having a rate similar to that achieved by capacity-approachingvariable-length code when used with non-active ACK/NACK feedback; and(j) wherein probability of either failing to decode a k-bit message, ordecoding a k-bit message in error, is below a target value.

17. The receiver apparatus of any preceding embodiment, wherein saidinitial transmission and said one or more increments are configured withlengths

_(l),

,

_(m), that have been optimized using sequential differentialapproximation.

18. The receiver apparatus of any preceding embodiment, wherein saidreceiver apparatus is configured for utilizing a cyclic redundancy checkto determine whether each variable length decoding attempt wassuccessful.

19. The receiver apparatus of any preceding embodiment, wherein saidincremental redundancy decoder is configured for separately decodingeach of said m−1 groups of increments associated with k-bit messages, sothat at stage j where j ∈{2,

, m}, said incremental redundancy decoder takes as input both a commonpool of incremental redundancy associated with increment j from saidvariable-length code applied to each of said L k-bit messages and thej^(th) increment of length

_(j) corresponding to said k-bit messages that have already beensuccessfully decoded to produce said increments each of length

_(j) corresponding to said k-bit messages that have not yet beensuccessfully decoded.

20. The receiver apparatus of any preceding embodiment, wherein saidincremental redundancy decoder is configured for jointly decoding someof said groups of m−1 increments associated with a k-bit message, sothat in one of its operations said incremental redundancy decoder takesas input both said common pool of incremental redundancy associated withs groups of L increments associated with s consecutive stages, havinglengths

_(j),

_(j+1),

_(j+s−1) and s increments with lengths

_(j),

_(j+1),

_(j+s−1) corresponding to said k-bit messages that have already beensuccessfully decoded to produce said s increments with lengths

_(j),

_(j+1),

_(j+s−1) corresponding to said k-bit messages that have not yet beensuccessfully decoded.

21. The receiver apparatus of any preceding embodiment, wherein saidreceiver apparatus is configured for use in an optical point-to-pointchannel.

22. The receiver apparatus of any preceding embodiment: furthercomprising an additional decoder configured for recovering any k-bitmessages not successfully decoded by said variable-length decoders; andwherein said additional decoder is configured for taking as input saiddecoded k-bit messages and locations of k-bit messages that could not bedecoded, and said additional decoder is configured for producing a fullset of L k-bit messages as a result, while failing only when too manyk-bit messages are not successfully decoded by said variable-lengthcodes, which occurs with a probability below a target value for thespecified point-to-point channel.

23. A method for communicating information at a rate approachingcapacity by exceeding at least a specified percentage of Shannoncapacity on a specified point-to-point channel, comprising the steps:(a) utilizing one or more variable-length encoders for a variable-lengthcode within a transmitter circuit that takes as input a k-bit messageand produces as output an initial transmission sequence of length

₁ symbols and one or more increments having lengths

₂,

,

_(m) symbols; (b) wherein, if said initial transmission and saidincrements were transmitted based on non-active ACK/NACK feedback, thensaid variable-length codes approach capacity by exceeding at least aspecified percentage of Shannon capacity of a specified point-to-pointchannel, without using feedback to control the transmission of saidincrements; (c) compressing and encoding increments produced by said oneor more variable-length encoders, which perform incremental redundancyencoding, to produce a common pool of incremental redundancy to be madeavailable for all variable-length codes at a receiver; (d) utilizingsaid one or more variable-length encoders within a transmitter circuitto produce initial transmissions responsive to a plurality L of saidk-bit messages, which are transmitted over said specified point-to-pointchannel, and transmitting over said specified point-to-point channelsymbols corresponding to said common pool of redundancy responsive tosaid plurality L of said k-bit messages; (e) wherein said number oftransmitted symbols corresponding to said common pool of redundancyapproximates said number of symbols of incremental redundancy that wouldhave been sent by L independently operating variable-length encoders ifsaid increments were transmitted based on non-active ACK/NACK feedbackon the specified point-to-point channel; (f) wherein a receiver circuitis configured for utilizing said one or more variable-length decodersfor performing multiple decoding attempts where each successive attemptuses as its input an input sequence of a previous attempt plus anadditional increment, and wherein said one or more variable-lengthdecoders is producing for each attempt either a decoded k-bit message,or an indication that no decoded k-bit message is available for thatattempt; (g) utilizing an incremental redundancy decoder taking asinput: (g)(i) received symbols corresponding to a common pool ofredundancy produced by an incremental redundancy encoder, and (g)(ii)increments produced by already-decoded k-bit messages and producing asoutput increments that can be used by said variable-length decoders; (h)first attempting to decode said plurality of L initial transmissionswithin said one or more variable-length decoders, each of said L initialtransmissions corresponding to a k-bit message, which are transmittedover said specified point-to-point channel, and continuing untildecoding is successful, by continuing to attempt decoding whenever anadditional increment is made available from said incremental redundancydecoder for said variable-length decoder for that k-bit message; (i)utilizing increments corresponding to each successfully-decoded k-bitmessage within said incremental redundancy decoder to subsequentlyproduce additional increments for said variable-length decoders until nomore new increments from successfully-decoded k-bit messages areavailable; (j) wherein said method approaches Shannon capacity of saidspecified point-to-point channel, having a rate similar to that achievedby capacity-approaching variable-length codes when used with non-activeACK/NACK feedback; and (k) wherein probability of either failing todecode a k-bit message, or decoding a k-bit message in error, is below atarget value.

24. The method of any preceding embodiment, wherein said initialtransmission and said one or more increments have lengths

₁,

,

_(m) that have been optimized using sequential differentialapproximation.

25. The method of any preceding embodiment, further comprising utilizingan additional encoder in a transmitter circuit, and an additionaldecoder in the receiver circuit, for protecting said k-bit messages sothat all user information can be recovered even if some k-bit messagesare not successfully decoded by said variable-length decoders, saidadditional encoder taking L×R_(e) k-bit messages containing userinformation as input and producing as output the L k-bit messagesencoded by said variable-length encoders so that probability that a fullset of L×R_(e) k-bit messages is not recovered successfully is below atarget value.

26. The method of any preceding embodiment, further comprising jointlyencoding at least a portion of said m−1 increments associated with ak-bit message in said incremental redundancy encoder, so that in one ofits operations said incremental redundancy encoder takes as input sgroups of L increments associated with s consecutive stages, havinglengths

_(j),

_(j+1),

_(j+s−1) to jointly produce a portion of said common pool of incrementalredundancy associated with increments j through j+s−1.

27. A transmitter apparatus in a communications system, said transmitterapparatus comprising: (a) one or more variable-length encoders for avariable-length code configured to take as input a k-bit message and toproduce as output an initial transmission sequence of length

₁ symbols and one or more increments having lengths

₂,

,

_(m) symbols, such that if said initial transmission sequence and saidincrements were transmitted based on non-active ACK/NACK feedback, saidvariable-length code would approach capacity by exceeding at least aspecified percentage of Shannon capacity of a specified point-to-pointchannel, without using feedback to control the transmission of saidincrements; and (b) an incremental redundancy encoder configured tocompress and encode increments produced by said one or morevariable-length encoders to produce a common pool of incrementalredundancy to be made available for all variable-length codes at areceiver; (c) wherein said transmitter apparatus is configured toutilize said one or more variable-length encoders to produce saidinitial transmission sequences of a plurality L of said k-bit messages,which are transmitted to a receiver over a channel between saidtransmitter and said receiver; (d) wherein said transmitter apparatus isconfigured to transmit to said receiver over said channel between saidtransmitter and said receiver a number of transmitted symbols,corresponding to a common pool of redundancy for the plurality L of saidk-bit messages; (e) wherein said number of transmitted symbolscorresponding to the common pool of redundancy approximates a number ofsymbols of incremental redundancy that would have been sent by Lindependently operating variable-length encoders if said increments weretransmitted based on non-active ACK/NACK feedback on said specifiedpoint-to-point channel; (f) wherein said transmitter apparatus isconfigured to approach Shannon capacity of a specified point-to-pointchannel, having a rate similar to that achieved by capacity-approachingvariable-length code when utilized with non-active ACK/NACK feedback;and (g) wherein probability of either failing to decode a k-bit message,or decoding a k-bit message in error is below a target value.

28. The transmitter apparatus of any preceding embodiment, wherein theinitial transmission has a length

₁ obtained using sequential differential approximation and the one ormore subsequent transmission sequences called increments all have thesame length

_(Δ), where the same length is determined by approximating the set oflengths obtained using sequential differential approximation by a singlelength.

29. The receiver apparatus of any preceding embodiment, wherein initialtransmission has a length

₁ obtained using sequential differential approximation and the one ormore subsequent transmission sequences called increments all have thesame length

_(Δ)where said same length is determined by approximating the set oflengths obtained using sequential differential approximation by a singlelength.

30. The transmitter apparatus of any preceding embodiment, whereins=m−1.

31. The receiver apparatus of any preceding embodiment, wherein s=m−1.

32. The transmitter apparatus of any preceding embodiment, whereinincremental redundancy encoder is a generalized low-density generatormatrix (LDGM) code that linearly combines increments with all variousdegrees of the systematic nodes described by a degree distribution thevarious degrees of the parity nodes described by a different degreedistribution.

33. The transmitter apparatus of any preceding embodiment, wherein theLDGM code is regular, with all systematic nodes having the same degreeand all parity nodes have the same degree.

34. The transmitter apparatus of any preceding embodiment, wherein theLDGM code is irregular, so that not all parity nodes have the samedegree.

35. The transmitter apparatus of any preceding embodiment, wherein thedegree distribution was obtained by differential evolution.

36. The transmitter apparatus of any preceding embodiment, wherein theincrement lengths

₂,

,

_(m) are all less than 20.

37. The transmitter apparatus of any preceding embodiment, wherein thevariable length code has an average blocklength of less than 150 symbolsis used with non-active ACK/NACK feedback on the specifiedpoint-to-point channel.

38. The transmitter apparatus of any preceding embodiment, wherein thevariable-length code achieves more than 75% of the Shannon capacity ofthe specified point-to-point channel.

39. The receiver apparatus of any preceding embodiment, wherein areliability function such as can be produced by the tail-bitingreliability-output Viterbi algorithm is used by the receiver todetermine whether each variable length decoding attempt was successful.

40. A method of high throughput communication with low frame error rates(FER), the method comprising: coupling a plurality of short blocklengtherror checking encoders in parallel, as parallel encoders, throughcommon incremental redundancy; coupling a plurality of short blocklengtherror checking decoders in parallel, as parallel decoders, throughcommon incremental redundancy; transmitting, at an encoder of atransmission side, a plurality of short blocklength code words inparallel to said parallel encoders; conveying encoded outputcommunication from said transmission side to a receiver side; decoding,at a decoder on a receiver side, a plurality of short blocklength codewords in parallel from each of said parallel decoders; receiving sideinformation for said decoding, from the parallel decoders that havealready successfully identified their codewords; and providingincremental redundancy to the parallel decoders that have not yetsuccessfully identified their codewords, in response to processing ofsaid side information.

41. The method of any preceding embodiment, wherein said error checkingcomprises parity checking.

42. The method of any preceding embodiment, wherein said parity checkingcomprises low density parity checking (LDPC).

43. The method of any preceding embodiment, wherein incrementalredundancy is generated without feedback to only select decoders thatrequire it.

44. The method of any preceding embodiment, wherein determining anamount of incremental redundancy to utilize in response to solvingcentral limit theorem arguments.

45. The method of any preceding embodiment, wherein said method isconfigured for harvesting the ergodicity benefits of long blocklengthswhile achieving low decoder complexity of a short-blocklength code byleveraging performance of short-blocklength codes with incrementalredundancy.

46. The method of any preceding embodiment, wherein said method performsencoding and decoding at a rate which can approach capacity at highthroughputs while permitting strong guarantees on frame error rate (FER)performance.

47. An apparatus for high throughput communication with low frame errorrates (FER), the apparatus comprising: an error checking encodercomprising a plurality of short blocklength error checking encoders inparallel, as parallel encoders, which are coupled through commonincremental redundancy; an error checking decoder comprising a pluralityof short blocklength error checking decoders in parallel, as paralleldecoders, which are coupled through common incremental redundancy;wherein communication is encoded at said error checking encoder, andconfigured for transmission to said error checking decoder; conveyingencoded output communication from said transmission side to a receiverside; wherein said error checking decoder is configured for decoding aplurality of short blocklength code words in parallel from each of itssaid parallel decoders; wherein receiving side information is utilizedin said decoding, from each of the parallel decoders that have alreadysuccessfully identified their codewords; and wherein incrementalredundancy is provided to the parallel decoders that have not yetsuccessfully identified their codewords, in response to processing ofsaid side information.

48. A method for high throughput communication with low frame errorrates (FER), the method comprising: using short-blocklength codes toachieve capacity with incremental redundancy; transmitting and decodinga large number of short-blocklength codewords in parallel; anddelivering incremental redundancy without feedback only to the decodersthat need incremental redundancy.

49. An apparatus for high throughput communication with low frame errorrates (FER), the apparatus comprising: an error checking encodercomprising a plurality of short blocklength error checking encoders inparallel, as parallel encoders, which are coupled through commonincremental redundancy; an error checking decoder comprising a pluralityof short blocklength error checking decoders in parallel, as paralleldecoders, which are coupled through common incremental redundancy;wherein short-blocklength codes are used to achieve capacity withincremental redundancy; wherein a large number of short-blocklengthcodewords are transmitted and decoders in parallel; and whereinincremental redundancy is delivered without feedback only to thedecoders that need incremental redundancy.

Although the description herein contains many details, these should notbe construed as limiting the scope of the disclosure but as merelyproviding illustrations of some of the presently preferred embodiments.Therefore, it will be appreciated that the scope of the disclosure fullyencompasses other embodiments which may become obvious to those skilledin the art.

In the claims, reference to an element in the singular is not intendedto mean “one and only one” unless explicitly so stated, but rather “oneor more.” All structural and functional equivalents to the elements ofthe disclosed embodiments that are known to those of ordinary skill inthe art are expressly incorporated herein by reference and are intendedto be encompassed by the present claims. Furthermore, no element,component, or method step in the present disclosure is intended to bededicated to the public regardless of whether the element, component, ormethod step is explicitly recited in the claims. No claim element hereinis to be construed as a “means plus function” element unless the elementis expressly recited using the phrase “means for”. No claim elementherein is to be construed as a “step plus function” element unless theelement is expressly recited using the phrase “step for”.

All cited publications and other references are incorporated herein byreference in their entireties.

TABLE 1 Parameters for m = 10, FER = 10⁻³, k = 280, with NB-LDPCcomputed using SDO Stage (j)

 (I_(j)) N_(j) P(i ∈ S_(j)) 1 428 428 0.0937 2 13 441 0.2142 3 11 4520.3555 4 10 462 0.4981 5 10 472 0.6352 6 11 483 0.7625 7 13 496 0.8705 816 512 0.9463 9 23 535 0.9878 10 49 584 0.9997

TABLE 2 Performance Comparison between VI and CI Designs 2 dB BI-AWGN, m= 5, 

 = 10⁻³, k = 64, 1024-state TBCC Simulated Transmission Length RateFailure Rate

 ₁, 

 

 ₅ (bits) R_(t) ^((FB)) δ(6) VI ES¹ 107, 9, 10, 13, 29 0.528756 9.42967× 10⁻⁴ SDO² 106, 10, 10, 13, 29 0.528655 9.42967 × 10⁻⁴ CI Best 107, 15,15, 15, 15 0.522791 9.26128 × 10⁻⁴ Actual 108, 16, 16, 16, 16 0.5208435.05167 × 10⁻⁴ ¹Exhaustive search; ²Sequential differential optimization

TABLE 3 Performance of the Feedforward Systems Rate Simulated PercentagePercentage R_(t) ^((FF)) Error Rate of Capacity of Feedback¹ Regular0.494503 6.691 × 10−4 77.01% 93.52% Irregular 0.510182 8.979 × 10−479.45% 96.48% ¹The percentage of R_(t) ^((FF)) w.r.t. the R_(t) ^((FB))of the best feedback system (the first row in Table 2).

1. A transmitter apparatus in a communications system, said transmitterapparatus comprising: (a) one or more variable-length encoders for avariable-length code configured to take as input a k-bit message and toproduce as output an initial transmission sequence of length

₁ symbols and a set of m−1 increments, where m is an integer greaterthan one, all increments having the same length

_(Δ), such that if said initial transmission sequence and saidincrements were transmitted based on non-active ACK/NACK feedback, saidvariable-length code would approach capacity by exceeding a specifiedpercentage of Shannon capacity of a specified point-to-point channel;(b) an incremental redundancy encoder configured to combine theincrements produced by said one or more variable-length encoders toproduce a common pool of incremental redundancy to be made available forall variable-length codes at a receiver; (c) wherein said one or morevariable-length encoders are configured to produce said initialtransmission sequence for a plurality L of said k-bit messages, whichare transmitted to a receiver over a channel between said transmitterand said receiver; (d) wherein said transmitter apparatus is configuredto additionally transmit to the receiver over said channel between saidtransmitter and the receiver a number of transmitted symbols less thanL×(m−1)×

_(Δ), corresponding to a common pool of incremental redundancy for theplurality L of said k-bit messages; (e) wherein said number oftransmitted symbols corresponding to the common pool of redundancyapproximates a number of symbols of incremental redundancy that wouldhave been sent by L independently operating variable-length encoders ifsaid increments were transmitted based on non-active ACK/NACK feedbackon said specified point-to-point channel; (f) wherein said transmitterapparatus is configured to have a rate approaching the rate achieved bythe capacity-approaching variable-length code when utilized withnon-active ACK/NACK feedback; and (g) wherein probability of eitherfailing to decode a k-bit message, or decoding a k-bit message in erroris below a target value; and— (h) wherein k is a number of bits in amessage that is encoded by a variable length encoder,

is a length in symbols of a transmission, m is a number of stagesincluding an initial transmission and m−1 additional stages ofincremental redundancy, and L is a number of short-blocklengthcodewords.
 2. The transmitter apparatus of claim 1, wherein said initialtransmission sequence of length

₁ is obtained by using sequential differential optimization and theincrement length

_(Δ) is obtained by approximating the set of lengths obtained bysequential differential optimization by a single length.
 3. Thetransmitter apparatus of claim 1: wherein the incremental redundancyencoder is a generalized low-density generator matrix (LDGM) code inwhich systematic nodes correspond to variable-length encoders/decodersand parity nodes correspond to linearly combined length-

_(Δ) increments; and wherein the systematic nodes have various degreesdescribed by a systematic-node degree distribution and the parity nodeshave various degrees described by a parity-node degree distribution. 4.The transmitter apparatus of claim 3, wherein all the systematic nodeshave the same degree, which is the number m−1 of increments produced byeach variable-length encoder.
 5. The transmitter apparatus of claim 3,wherein all the parity nodes have the same degree.
 6. The transmitterapparatus of claim 3, wherein all the parity nodes are of a sufficientlyclose degree that parity node degrees are one of two consecutiveintegers.
 7. The transmitter apparatus of claim 3, wherein said paritynode degree distribution is obtained by differential evolution.
 8. Thetransmitter apparatus of claim 1, wherein the increment length

_(Δ) is less than
 20. 9. The transmitter apparatus of claim 1, whereinthe variable length code has an average blocklength of less than 150symbols if used with non-active ACK/NACK feedback on the specifiedpoint-to-point channel.
 10. The transmitter apparatus of claim 1,wherein said transmitter apparatus is configured for being utilized inan optical point-to-point channel.
 11. The transmitter apparatus ofclaim 1, wherein said plurality L of k-bit messages includes redundantk-bit messages resulting from an additional encoder that allows recoveryof k-bit messages not successfully decoded and/or allows recovery ofk-bit incorrectly decoded by said variable-length decoders.
 12. Areceiver apparatus in a communications system, said receiver apparatuscomprising: (a) one or more variable-length decoders configured forreceiving a variable-length code where variable-length codewords arecomprised of an initial transmission sequence of length

₁ symbols by itself, or together with a set of up to m−1 increments,where m is an integer greater than one, all increments having the samelength

_(Δ), such that if said initial transmission and said increments weretransmitted based on non-active ACK/NACK feedback, said variable-lengthcode would approach capacity by exceeding a specified percentage ofShannon capacity of a specified point-to-point channel; (b) wherein saidvariable-length decoders are configured for performing multiple decodingattempts, in which each successive attempt utilizes as its input aninput sequence of a previous attempt plus one or more additionalincrements; (c) wherein said one or more variable-length decoders isconfigured for producing for each attempt either a decoded k-bit messageor an indication that no decoded k-bit message is available for thatattempt; and (d) an incremental redundancy decoder configured forreceiving as input (d)(i) received symbols corresponding to a commonpool of redundancy produced by an incremental redundancy encoder, and(d)(ii) increments produced by already-decoded k-bit messages, and saidincremental redundancy decoder is configured for producing outputincrements for use by said variable-length decoders; (e) wherein saidone or more variable-length decoders is configured to first attemptdecoding of said plurality of L initial transmissions, eachcorresponding to a k-bit message, which are transmitted over a channelbetween the transmitter and the receiver, and continue until decoding issuccessful or no more increments are available; (f) wherein said one ormore variable-length decoders is configured for continuing to attemptdecoding by a variable-length decoder whenever an additional incrementis made available from said incremental redundancy decoder for saidvariable-length decoder for that k-bit message; (g) wherein saidincremental redundancy decoder is configured for utilizing incrementscorresponding to each successfully-decoded k-bit message to subsequentlyproduce additional increments for said variable-length decoders until nomore new increments from successfully-decoded k-bit messages areavailable; (h) wherein said number of received symbols received by theincremental redundancy decoder is less than L×(m−1)×

_(Δ), corresponding to the common pool of incremental redundancy, whichapproximates said number of symbols of incremental redundancy that wouldhave been sent by L independently operating variable-length encoders ifsaid increments were received based on non-active ACK/NACK feedback froma receiver on that same channel; (i) wherein a communications systemutilizing said receiver apparatus approaches Shannon capacity of saidspecified point-to-point channel, having a rate similar to that achievedby capacity-approaching variable-length code when used with non-activeACK/NACK feedback; (j) wherein probability of either failing to decode ak-bit message, or decoding a k-bit message in error, is below a targetvalue; and (k) wherein k is a number of bits in a message that isencoded by a variable length encoder,

is a length in symbols of a transmission, m is a number of stagesincluding an initial transmission and m−1 additional stages ofincremental redundancy, and L is a number of short-blocklengthcodewords.
 13. The receiver apparatus of claim 12: wherein theincremental redundancy decoder is a decoder for a generalizedlow-density generator matrix (LDGM) code where the systematic nodescorrespond to variable-length encoders/decoders and the parity nodescorrespond to linearly combined length-

_(Δ) increments; and wherein the systematic nodes have various degreesdescribed by a systematic-node degree distribution and the parity nodeshave various degrees described by a parity-node degree distribution. 14.The receiver apparatus of claim 13, wherein all the systematic nodeshave the same degree, which is the number m−1 of increments produced byeach variable-length encoder.
 15. The receiver apparatus of claim 13,wherein all the parity nodes have the same degree.
 16. The receiverapparatus of claim 13, wherein all the parity nodes are of asufficiently close degree that parity node degrees are one of twoconsecutive integers.
 17. The receiver apparatus of claim 12, whereinsaid receiver apparatus is configured for utilizing a cyclic redundancycheck to determine whether each variable length decoding attempt wassuccessful.
 18. The receiver apparatus of claim 12, wherein saidvariable length code from said transmitter is configured for utilizing areliability function as produced by a reliability-output Viterbialgorithm in a receiver to determine whether each variable-lengthdecoding attempt was successful.
 19. The receiver apparatus of claim 12,wherein said receiver apparatus is configured for use in an opticalpoint-to-point channel.
 20. The receiver apparatus of claim 12: furthercomprising an additional decoder configured for recovering any k-bitmessages not successfully decoded by said variable-length decoders; andwherein said additional decoder is configured for taking as input saiddecoded k-bit messages and locations of k-bit messages that could not bedecoded, and said additional decoder is configured for producing a fullset of L k-bit messages as a result, while failing only when too manyk-bit messages are not successfully decoded by said variable-lengthcodes, which occurs with a probability below a target value for thespecified point-to-point channel.